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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  is  examining 
the  use  of  distance  learning  technologies  for  use  by  soldiers  in  an  “on  demand”  environment. 

This  research  under  the  TRAINTODAY  project,  sponsored  by  the  Training  and  Doctrine 
Command  (TRADOC),  seeks  to  provide  guidance  to  the  Army  as  it  transforms  from  a 
classroom-centric  method  of  instruction  to  one  that  is  more  soldier-centric  and  collaborative 
rather  than  classroom  based. 

In  recent  years,  the  field  of  educational  technology  has  witnessed  the  emergence  of  many 
e-leaming  tools  (tools  for  instruction  that  use  the  Internet)  as  well  as  many  collaborative  learning 
environments  for  online  instruction.  As  the  TRAINTODAY  project  shifts  focus  to  Web-based 
collaborative  learning,  experimental  approaches  need  to  be  determined.  At  the  same  time,  the 
many  results  and  methods  developed  in  education  need  to  be  considered  for  inclusion  in  the 
Army’s  future  plans.  Strategies  to  adapt  the  best  practices  from  education  to  military  training 
need  to  be  pursued.  The  research  framework  for  these  plans  and  strategies  was  presented  to  the 
Training  Development  and  Analysis  Directorate,  TRADOC  on  2  August  2000. 
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Applying  Collaborative  and  e-Learning  Tools  to  Military  Distance  Learning:  A  Research 
Framework 


EXECUTIVE  SUMMARY _ 

Research  Requirement: 

The  Army  is  embarking  on  a  transformation  to  deliver  standardized  individual,  self¬ 
development,  and  small  group  training  to  soldiers  through  the  application  of  networked 
communication  systems.  Army  training  will  become  more  learner-centric  with  soldiers 
assuming  increased  responsibility  for  the  acquisition  of  knowledge  and  the  development  of 
skills.  The  documentation  of  approaches  to  learning  and  the  tools  needed  for  such  a  learner¬ 
centric  paradigm  has  been  much  more  extensive  in  educational  rather  than  training  settings. 
Because  of  differences  in  the  goals,  outcomes,  and  eventual  performance  applications  of  what 
was  learned,  education  and  training  have  fundamental  differences.  One  cannot  assume  that  what 
works  well  in  education  will  necessarily  work  well  in  training.  An  examination  of  this 
educational  literature  along  with  a  research  framework  on  how  to  adapt  the  benefits  of 
collaboration  and  e-learning  to  military  training  in  a  soldier-centric  paradigm  is  needed. 

Procedure: 

Database  searches  yielded  over  230  relevant  reports,  80  percent  published  since  1996. 
Summaries  of  findings  on  collaborative  tools,  individual  differences,  and  learning  communities 
are  provided  and  gaps  in  the  research  literature  are  identified.  A  set  of  experiments  derived  from 
this  literature  are  designed  to  evaluate  the  adaptation  of  research  findings  from  the  educational 
literature  to  an  Army  training  context . 

Findings: 

Educational  research  literature  points  to  clear  research  directions  for  learner-centric 
approaches  within  military  training.  The  training  approaches  derive  from  psychological 
principles  in  cognition,  motivation,  social  factors,  and  individual  differences.  The  key  research 
areas  include  instructor’s  roles  in  online  training,  online  moderators,  learner  perceptions, 
methods  for  online  collaboration,  interaction  schemes,  collaborative  tools,  online  communities, 
and  learning  styles.  Ten  primary  experiments  are  profiled  in  terms  of  theory,  independent  and 
dependent  variables,  hypotheses,  and  follow-up  training  activities. 

Utilization  of  Findings: 

The  proposed  experiments  can  be  implemented  in  a  number  of  ways.  For  example,  the 
experiments  can  be  embedded  in  courses  undergoing  a  transition  from  the  classroom  to  a 
distributed  learning  environment.  The  experiments  can  be  applied  to  either  professional 
development  education  or  specialized  skill  training  throughout  the  military.  The  conduct  of 
several  primary  experiments  is  planned  as  part  of  continuing  research  on  training  tools  in  web- 
based  and  collaborative  environments  by  the  U.S.  Army  Research  Institute. 


vii 


Applying  Collaborative  and  e-Leaming  Tools  to  Military  Distance  Learning: 
A  Research  Framework 


Contents _ _ _ 

Introduction . 1 

Distributed  Learning  Background . 2 

Education  versus  Military  Training . 3 

The  Emergence  of  E-Leaming  Technologies . 4 

Constructivism . 6 

Learner-Centered  Principles . 7 

Web-Based  Learning  Models . 9 

Learning  Team  Centered  Approach . 10 

New  Role  for  Instructors  Online . 11 

Guidelines  for  Facilitation . 11 

Online  Moderator  Research . 13 

Online  Collaboration . 14 

Research  Methodologies  in  Collaborative  Environments . 15 

Content  Analysis . 10 

Collaborative  Tool  Interaction  Research . 18 

Learning  Styles . 28 

Online  Communities . 29 

Community  Membership . 30 

Online  Learning  Community  Examples . 33 

Online  Classes  as  Learning  Communities . 33 

Other  Variables  of  Interest . 35 

Research  on  Internet-Based  Distance  Learning . 36 

E-Leaming  Tools . 36 

Collaborative  Tools  in  Distance  Learning . 38 

Educational  Websites . 38 

Survey  and  Evaluation  Tools . 39 

Research  Gaps . 41 

Ten  Primary  Experiments . 41 

Experiment!.  Online  Discussion:  Online  Officer  Training  Program . 42 

Experiment  II.  Variations  in  Instructor  Moderation:  E-Leaming  Environments . 43 

Experiment  III.  Student  Perceptions  of  E-Leaming  Environment . 44 

Experiment  IV.  The  Development  of  Online  Learning  Communities . 45 

Experiment  V.  Time  Logging:  Effect  of  E-Leaming  Approach  on  Time . 46 


Experiment  VI.  Critical  Thinking  and  Problem  Solving  Applications  in  Synchronous  and 
Asynchronous  Environments . 47 

Experiment  VII.  Peer  Tutoring  and  Online  Mentoring:  Impact  of  Peer  Tutoring  on  E- 
Leaming  Achievement . 48 

Experiment  VIII.  Student  Retention:  E-Leaming  and  Attrition . 49 

Experiment  IX.  Conceptual  Referencing:  Collaboration  and  Explicit  Referencing . 50 

Experiment  X.  Online  Collaboration:  Group  Performance  and  Collaborative  Learning 
Technologies . 

Suggested  Secondary  Experiments . 52 

New  Tools  and  Pedagogies . 52 

References . 54 

APPENDIX  A.  Supplemental  Tables . A'1 

B.  List  of  URLs  (Universal  Resource  Locators)  cited  in  text . B-l 

C.  Secondary  Experiments . C-l 

LIST  OF  TABLES 

Table  1 

Learner-Centered  Psychological  Principles  Revised . 8,  A-l 

Table  2 

Sequence  of  steps  within  a  virtual  debate  (Cummings,  2000) . 12 

Table  3 

Interaction  Process  Analysis  Categories  (Bales,  1950  as  cited  in  Hiltz  et  al.,  1986) . 19 

Table  4 

Uses  of  Online  Conferencing  (adapted  and  modified  from  Selinger,  1997) . 20 

Table  5 

Interactions  Found  in  Distance  Learning . 22 

Table  6 

Model  for  Analyzing  Online  Processing  of  Information  (adapted  and  modified 

from  Henri,  1992;  see  also  Hara  et  al.,  2000) . 22 


x 


Table  7 

Indicators  of  critical  thinking  for  online  discourse  (from  Newman,  Johnson, 
Cochrane,  &  Webb,  1996) . 


Constructivist  Interaction  Analysis  Coding  Scheme  (Kanuka  &  Anderson,  1998) 


Table  9 


Coding  scheme  to  describe  utterances  in  online  collaboration  (Curtis  &  Lawson,  1999) . A-4 


Table  10 


Cognitive  and  interactive  acts  in  online  discourse  (Bakardjieva  &  Harasim,  1997). 
Table  11 

Twelve  forms  of  electronic  learning  mentoring  and  assistance 
(Bonk  &  Kim,  1998;  Tharp,  1993) . 


.26 


.A-5 


Social  Constructivism  and  Learning  Communities  Online  (SCALCO)  Scale 
(Bonk,  Malikowski,  Angeli,  &  East,  1998;  Bonk,  Oyer,  &  Medury,  1995; 
Hanuka  &  Anderson,  1998) . 


xi 


Applying  Collaborative  and  e-Learning  Tools  to  Military  Distance  Learning. 

A  Research  Framework 

Introduction 

For  reasons  of  cost  and  accessibility,  the  Army  has  a  growing  need  for  training  to  be 
delivered  in  the  future  on  an  “anywhere  at  anytime”  basis  through  distributed  training 
technologies  (TRADOC,  1999).  Towards  this  end,  the  Training  and  Doctrine  Command  is 
transforming  courses  and  classrooms  to  accommodate  the  distributed  training  concept.  The 
National  Guard  Bureau  established  the  Distributive  Training  Technology  Project,  which  provides 
high  network  links  to  armories  in  all  states  and  territories.  The  Army  Reserve  has  a  Distance 
Learning  Futures  Group  examining  alternatives  to  the  traditional  classroom  training  model.  The 
pedagogical  methods  and  instructional  technologies  supporting  such  a  transformation  must  be 
tailored  to  this  future  training  environment,  which  will  certainly  include  new  and  creative 
approaches  for  soldiers  to  learn. 

The  Internet  is  advancing  the  creation  of  engaging  e-leaming  tools  that  transcend  typical 
time  and  space  barriers.  E-leaming  tools  refer  to  Internet-based  programs  designed  for 
instructional  purposes,  such  as  interactive  multimedia  displays  or  threaded  electronic  messaging. 
Web-based  collaborative  environments  are  a  special  category  of  e-leaming  tools  that  support  a 
group  of  learners  in  achieving  a  common  learning  goal.  Both  have  been  successfully  established 
in  educational  settings  from  K-12  to  higher  education  (Bonk  &  King,  1998).  Within  the  military, 
collaborative  environments  have  been  used  for  training  collective  skills  over  high-speed 
communication  networks  linking  remote  simulators  (Shlechter,  Bessemer  &  Kolosh,  1992). 


Little  systematic  research,  however,  has  been  conducted  in  applying  the  new  genre  of  e- 
leaming  tools  to  military  training.  This  report  presents  a  framework  for  applying  these  new 
educational  technologies  to  military  training  settings.  The  framework,  as  described  by  a  series  of 
experiments,  can  serve  as  a  basis  for  developing  and  evaluating  Web-based  approaches  to  small- 
group  instruction  and  individual  training  in  the  Army1. 

Systematic  research  on  collaborative  learning  and  online  tools  has  flourished  due  to 
funding  from  the  National  Science  Foundation,  the  Department  of  Education,  numerous 
foundations,  national  associations,  and  intramural  university  support.  Naturally,  the  focus  has 
been  on  educational  results  rather  than  training  outcomes.  A  central  issue  is  whether  the  benefits 
of  Web-based  instruction  reported  in  the  educational  literature  can  be  adapted  to  the  Army’s  new 
training  delivery  strategy.  The  Army  should  consider  leveraging  successful  research  and 
instructional  practices  found  “anyplace  anyhow”  to  meet  its  future  “anytime  anywhere”  training 
strategy.  As  described  later,  caution  should  be  taken  when  assuming  that  what  works  well  in 
education  will  necessarily  work  as  well  in  training.  To  gain  the  full  benefits  of  Web-based 
instruction,  a  broad  examination  of  the  educational  literature  on  e-leaming  tools  is  needed. 


'There  are  certainly  examples  of  educational  settings  in  the  military,  such  as  those  related  to 
professional  development  education,  which  account  for  6%  of  the  military  training  load. 
However,  the  large  investment  is  in  training  specialized  skills,  the  remaining  94%  of  the 
training  load  (Department  of  Defense,  1999). 
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Alongside  this  examination  should  be  a  framework  for  transforming  what  has  worked  in 
education  to  what  can  work  in  the  Army.  This  report  addresses  these  needs. 


Report  Overview 

This  report  is  intended  as  a  source  guide  for  those  involved  with  planning,  evaluating, 
and  implementing  e-leaming  and  collaborative  tools  in  military  environments.  It  is  particularly 
suited  for  those  who  require  a  broad  review  of  recent  advances.  This  report  focuses  on 
opportunities  for  e-leaming  application  rather  than  on  analyses  of  specific  Army  training  issues, 
the  impetus  being  the  Army’s  transformation  of  training  to  a  distributed  learning  enterprise. 
Generic  examples  of  how  these  technologies  can  be  applied  to  specific  training  issues  in  Army 
settings  are  offered. 

The  report  discusses  the  state  of  Web-based  training  and  online  collaboration  from  a 
learner-centered  psychological  perspective.  Next,  it  summarizes  the  research  on  online  learning 
and  simultaneously  identifies  key  areas  to  target  for  research.  The  review  of  the  extensive 
literature  on  e-leaming  technologies  and  collaborative  tools  in  the  educational  arena  is  intended 
to  be  in  depth  and  focused  on  recent  advances  -  of  the  over  230  citations,  nearly  80  percent  were 
published  between  1996  and  2000.  This  section  ends  with  a  description  of  the  key  components 
of  electronic  learning  communities.  Once  the  research  is  summarized  and  gaps  identified, 
experiments  are  suggested  for  military  training  settings. 

As  a  note  to  the  organization,  six  lengthy  tables  have  been  moved  from  the  body  to 
Appendix  A  in  the  interest  of  readability.  Also,  all  URL  links  have  been  consolidated  in 
Appendix  B,  with  reference  to  the  appropriate  page  number. 

Distributed  Learning  Background 

A  marked  shift  in  military  training  is  underway.  The  Training  and  Doctrine  Command 
(TRADOC)  is  embarking  on  a  major  change  to  deliver  standardized  individual  and  self¬ 
development  training  to  soldiers  through  the  application  of  multiple  media  and  networked 
delivery  technologies.  Training  is  to  move  from  a  classroom-centric  delivery  of  instruction  to  a 
learner-centric  model,  in  which  soldiers  assume  greater  responsibility  for  learning  facts, 
procedures,  and  complex  skills  as  well  as  teamwork  skills.  In  concert  with  this  paradigmatic 
shift,  the  Army  Research  Institute  is  pursuing  advanced  research  on  Web-based  instructional 
methods  and  learning  strategies  that  can  make  training  available  in  either  the  workplace,  at 
soldiers’  residences,  or  other  alternatives  to  the  traditional  classroom. 

Related  to  the  Army’s  change  in  training  delivery,  the  Department  of  Defense  (DoD)  has 
established  the  Advanced  Distributed  Learning  (ADL)  initiative.  This  initiative  grew  out  of  the 
DoD  strategy  to  “harness  the  power  of  learning  and  information  technologies  to  modernize 
education  and  training”  (DUSD  (R),  1999).  This  initiative  capitalizes  on  emerging  network 
technologies  to  tie  together  distributed  instructional  resources,  including  intelligent  tutors, 
subject  matter  experts,  and  traditional  instruction  to  support  learner-centric  education  on  a 
continuing  basis.  The  ADL  initiative  also  marks  a  shift  from  the  current  classroom  and  distance 
teaching  philosophy  to  a  model  of  anytime,  anywhere  learning. 
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ADL  Workshop.  A  front-end  assessment  framing  ADL  research  issues  was  conducted  at 
a  four-day  ADL  Science  and  Technology  Workshop  held  in  October  1999.  An  outcome  of  that 
workshop  was  a  vision  that  “ADL  in  2012  ”  will  be  collaborative,  affordable,  and  adaptive 
instructional  environments.  Individuals  and  teams  will  be  supported  by  a  system  that  promotes 
competencies  such  as  problem  solving,  analysis,  evaluation,  reasoning,  and  decision  making. 
They  will  be  supported  by  an  instructor  and  peer-based  dynamic  mentoring  environment.  One  ol 
the  key  research  areas  identified  during  the  workshop  concerned  collaborative,  group,  and  team 
learning  The  objective  of  such  research  is  the  development  of  mechanisms  to  enhance 
instructional  effectiveness  of  learner-learner  collaborations,  learner-instructor  interactions,  and  to 
promote  team-building  skills.  Key  technical  areas  are  an  understanding  of  the  role  of  interaction 
and  collaboration  in  learning  and  a  definition  of  models  for  collaboration  and  interaction 
considering  distance,  content,  roles,  and  task  requirements.  The  research  framework  outline  m 
the  present  report  is  compatible  with  these  ADL  research  goals. 

As  we  enter  the  21st  century  innovative  learning  tools  for  education  and  training  continue 
to  evolve  and  expand.  Virtual  communities,  distributed  simulations,  virtual  realities,  online 
visualization  tools,  intelligent  agents,  and  technology  for  displaying  complex  patterns  of 
knowledge  have  all  reconceptualized  distance  learning  environments  during  the  past  decade 
(Dede,  1996a).  The  proliferation  of  Web  courseware  technologies  and  collaborative  tools 
multiply  the  opportunities  and  challenges  facing  higher  education  as  well  as  training 
environments  (Gray,  1 999).  Learning  will  be  seen  as  more  socially  shared,  active,  and 
interactive  than  in  the  past.  In  fact,  Dede  (1996a,  p.  29)  argues  that  “education  must  help  all 
students  become  adept  at  distance  interaction  because  skills  involving  information-gathering 
from  remote  sources  and  collaborating  with  dispersed  team  members  are  as  central  to  the  future 
workplace  as  learning  to  perform  structured  tasks  quickly  was  to  the  industrial  revolution. 

Since  the  preponderance  of  research  has  been  oriented  to  educational  rather  than  training 
settings,  so  a  brief  review  of  the  distinction  is  needed. 


Education  versus  Military  Training 

Education  and  training  share  the  psychological  constructs  of  learning,  memory  and 
motivation.  However,  fundamental  differences  in  the  goals,  outcomes,  and  eventual  application 
of  the  underlying  instruction  distinguish  the  two.  Learning  outcomes  are  measures  of  the 
knowledge  gained  from  an  instructional  program.  In  education,  which  has  historically  been 
concerned  with  the  social  and  intellectual  development  of  the  whole  person,  there  is  no  upper 
limit  to  how  elevated  a  learning  outcome  should  be.  The  range  of  a  learning  outcome  is 

generally  open  ended.  The  thought  of  producing  a  student  who  is  “over  educated  is 

inconceivable  to  an  educational  provider.  In  contrast,  the  thought  of  a  student  being  over  trained 
can  be  costly,  in  terms  of  lime  and  money,  to  a  training  provider.  It  is  better  to  have  the  prepared 
student  productive  on  the  job  rather  than  remain  in  a  classroom. 

Military  training  is  concerned  with  increasing  the  capacity  to  perform  military  functions 
and  tasks  (Dept,  of  Army,  1 990).  For  training  specialized  skills  in  the  military,  learning 
outcomes  are  established  by  doctrine  and  the  criteria  for  acceptable  proficiency  are  generally 
fixed  For  example,  soldiers  are  often  rated  on  the  dichotomous  Go/No-Go  scale  on  tasks  during 
formal  training.  The  immediate  goal  of  training  specialized  skills,  then,  is  moving  a  trainee  from 
ground  zero  to  at  least  the  acceptable  criterion.  Once  the  initial  level  of  proficiency  is  achieved, 
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learning  can  continue  in  a  workplace  context  during  follow-on  assignments.  Learning  beyond 
the  criterion  during  formal  training  is  nice  but  not  always  necessary^. 

Although  a  new  instructional  practice  might  succeed  in  an  educational  setting,  there  is  no 
guarantee  of  its  success  in  a  military  training  environment.  Differences  in  terms  of  learning 
cultures,  social  interactions,  motivational,  and  affective  factors  can  influence  the  transfer 
potential  between  environments  (Seidel,  1994).  Moreover,  the  fundamental  distinction  between 
education  and  training,  the  former  focused  on  the  open  acquisition  of  knowledge  and  the  latter 
focused  on  improving  job  performance,  must  also  be  considered.  The  benefits  of  innovative 
approaches  to  instruction  can  be  ultimately  determined  through  research  that  adapts  and 
evaluates  the  new  collaborative  and  e-leaming  tools  in  Army  settings. 

As  training  in  military  settings  becomes  increasingly  distributed  through  the  Internet  or 
intranets,  it  is  advantageous  to  understand  the  documented  Web-enabled  instructional  approaches 
and  learning  results.  Key  military  reports  advocate  research  on  interaction  and  collaboration  in 
learning,  collaborative  activities  and  interaction  strategies,  the  effectiveness  of  team-level 
tutoring,  and  associated  models  for  this  collaboration  (DUSD(R),  1999).  Such  reports  also 
discuss  the  need  to  establish  more  learner-centered,  constructivist  training  environments. 

The  Army  has  a  vested  interest  in  collaborative  learning  and  is  initiating  several 
applications.  For  example,  the  Depth  and  Simultaneous  Attack  Battle  Lab  (D&SABL)  at  the 
Field  Artillery  School  is  witnessing  a  tremendous  influx  of  automated,  information  handling,  and 
decision  support  systems  designed  to  support  collaborative  operations  between  military  decision¬ 
makers  and  the  staff  elements  (Siegel,  Burton,  Barnette,  Ross,  Ross,  &  Klinger,  2000).  In 
determining  the  requirements  for  a  collaborative  system,  the  D&SABL  is  examining  information 
requirements,  information  flow,  and  situational  awareness  in  synthetic  task  environments.  Pilot 
studies  are  underway  examining  collaborative  environments  in  these  training  contexts.  In 
another  example  at  the  U.S.  Army  Armor  School,  the  use  of  collaborative  learning  environments 
is  being  tested  (with  National  Guard  soldiers)  in  the  “synchronous  remote”  phase  of  the  Armor 
Captains  Career  Course.  Students  at  remote  sites  collaborate  through  role  playing  of  various 
staff  positions  in  the  venue  of  a  virtual  tactical  operations  center.  Collaboration  occurs  both 
through  a  live  audio  connection  and  online  chats  with  the  use  of  added  collaborative  learning 
tools  proposed  for  future  iterations  of  the  course.  This  course  is  a  candidate  for  one  of  the  initial 
experiments  proposed  later  in  this  report. 


The  Emergence  of  E-Learning  Technologies 

This  section  shifts  focus  to  the  growth  of  interest  in  e-leaming  technologies.  The  concept 
of  constructivism,  which  underpins  many  of  the  recent  advances  in  e-leaming  and  collaborative 
environments,  is  described.  An  accounting  of  learner-centered  principles  in  psychology  is 
presented.  These  principles  should  be  of  concern  to  those  involved  with  the  shift  from  a 


:  One  method  for  improving  skill  retention  is  to  maximize  the  amount  of  original  learning  (Wisher,  Sabol  &  Ellis, 
1999).  Learning  beyond  the  fixed  criteria  can  be  useful  for  skill  retention;  if  it  is  important  to  do  so,  then  the  criteria 
should  be  adjusted. 
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classroom  model  to  a  soldier-centric  model  of  training.  A  lengthy  review  of  specific  findings  in 
the  educational  field  then  sets  the  stage  for  the  experimental  framework  in  training. 

There  is  a  myriad  of  reports  in  response  to  the  current  wave  of  online  teaching  and 
e-leaming  in  education.  Some  speak  to  the  increase  in  online  course  offerings  as  well  as 
universities  involved  in  e-leaming.  In  fact,  there  are  already  more  than  50,000  courses  taught 
online  and  1,000  universities  developing  and  offering  such  courses  (Camavale,  2000;  National 
Center  for  Education  Statistics,  1999).  Others  discuss  the  costs  or  obstacles  for  taking  such 
courses  as  well  as  the  forms  of  resistance  to  such  courses  and  programs  at  both  the  institutional 
and  individual  level  (Jaffee,  1998).  Still  others  are  focusing  on  key  market  trends,  social 
demographics,  stakeholders,  policy  makers,  major  players,  and  workplace  needs  (Cronin  & 
Duffy,  1997;  Upitis,  1999). 

One  such  report  from  the  TeleLeaming  Network  Centers  of  Excellence  (TeleLeaming 
NCE)  of  Canada  compared  eight  key  post-secondary  institutions  offering  e-leaming.  They  also 
provided  a  preliminary  analysis  of  universities  emerging  in  this  field  (Massey  &  Curry,  1999). 
Importantly,  the  TeleLeaming  report  contained  a  competitive  analysis  of  the  courses/programs, 
pedagogy,  and  learner  support  in  place  at  each  of  these  institutions.  In  addition,  it  addressed 
expansion  plans,  marketing,  faculty,  leamers/clients,  and  course  production  and  delivery.  As 
such,  this  particular  report  provided  useful  insights  into  the  direction  of  online  technologies  and 
course  delivery. 

Web-based  distance  learning  is  a  growing  field  with  rapid  changes.  Why  the  flurry? 

One  reason  is  the  shelf  life  of  technical  skills  is  now  about  five  years  (Oblinger  &  Maruyama, 
1996).  Continual  reskilling  is  a  fact  of  life.  In  addition,  according  to  Oblinger  and  Maruyama, 
students  are  more  diverse  than  in  the  past;  they  are  often  older  students  who  prefer  to  attend 
college  on  a  part-time  basis.  “Internet  courses  have  clearly  emerged  as  the  technology-of-choice 
for  part-time  adult  students  who  cannot  physically  attend  classes,  either  because  of  situational  or 
dispositional  barriers”  (Edelson,  1998,  p.  3).  Peter  Drucker’s  prediction  that  universities  may  not 
survive  the  next  30  years  may  be  bold  and  overstated,  but  the  Web  has  definitely  opened  up  new 
options  for  students.  Nearly  all  Fortune  1000  companies  already  offer  some  type  of  computer- 
based  training  online,  and,  soon,  almost  every  major  university  will  offer  at  least  some  of  its’ 
courses  online  (H either,  1997).  The  emergence  of  the  corporate  university  to  provide  an 
enterprise-wide  strategy  for  online  learning  is  a  recent  national  trend  (Kenyon,  1999).  With  such 
rapid  changes,  there  is  much  uncertainty  and  controversy  of  opinions  about  the  directions  of 
distance  learning. 

E-leaming  is  becoming  more  accepted  and  expected  both  in  training  and  formal 
education  environments  (Hall,  1997).  However,  according  to  Besser  and  Bonn  (1997),  some 
great  success  stories  in  the  early  paradigms  of  distance  learning  have  come  from  the  field  of 
training,  not  education.  This  might  be  explained  by  the  fact  that  communicating  specific  skills 
and  training  them  to  a  fixed  standard  may  be  easier  in  a  distance  learning  environment.  Despite 
the  rapid  adoption  by  business  and  industry,  it  is  difficult  to  find  research  on  e-leaming  within 
training  environments  that  attempts  to  answer  some  of  the  critical  questions:  What  are  the 
advantages  of  such  technologies  in  corporate,  governmental,  and  military  settings?  How  can 
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effectiveness  be  measured?  What  are  powerful  pedagogical  approaches  for  online  learning  in  the 
workplace? 

There  are  predictions  that  Web-based  training  will  grow  tremendously  in  adult  and 
vocational  education  settings  as  well  as  in  corporate  training  sectors  (Imel,  1997;  Phillips,  1998). 
Trends  in  military  training  often  parallel  those  experienced  in  their  counterparts  in  corporate  and 
higher  education  settings.  For  instance,  as  with  most  workplace  and  college  classroom  settings, 
there  are  increasing  demands  within  the  military  for  learner-centered  instruction  and  the  use  of 
technology  tools  for  e-leaming,  especially  those  tools  that  attempt  to  foster  collaboration  (U.S. 
Army  Research  Institute,  1999;  Appendix  B). 

Partially  in  response  to  these  trends,  the  Army  also  plans  to  convert  525  courses  to 
distance  learning  formats  for  delivering  training  on  demand  (TRADOC,  1999).  This  is  a 
substantial  undertaking  requiring  careful  thought  and  experimentation.  Already  there  are 
experiments  to  use  the  Web  for  role  play,  such  as  war  simulations  and  case-based  crises  within 
military  training  (Campbell,  1997;  Comeaux,  Huber,  Kasprzak,  &  Nixon,  1998).  Virtual 
battlefields  have  been  created  to  develop  military  skills  such  as  decision  making  during  virtual 
battles  and  maneuvers  (Dede,  1996a). 

To  address  some  of  the  issues  noted  above,  the  ADL  initiative  described  earlier  is 
intended  to  utilize  emerging  network  technologies,  foster  collaboration,  control  training  costs, 
and  facilitate  the  development  of  needed  technologies  (DUSD(R),  1999).  This  push  coincides 
with  the  development  of  a  fourth  generation  of  distance  learning  technologies  rich  in 
collaboration,  multimedia,  and  interaction  (National  Center  for  Education  Statistics,  1999). 
Technology  tools  for  chat  sessions,  virtual  whiteboards,  application  sharing,  desktop 
videoconferencing,  computer  telephony,  multi-user  simulation  environments,  asynchronous 
communication,  and  audiographics  will  push  the  envelope  of  online  training  to  a  more 
collaborative  and  team  oriented  framework  than  in  the  past.  Such  interaction  points  also  to  the 
growing  importance  of  collaboration,  reflection,  critical  thinking,  evaluation,  and  decision 
making  skills,  instead  of  low-level  factual  knowledge  (Bonk  &  Kim,  1998;  Wisher,  et  al.,  1999). 
As  Web  technologies  proliferate,  skills  in  searching,  discovering,  filtering,  integrating,  and 
disseminating  knowledge  are  vital. 

Constructivism 

Prominent  reports  from  educational  researchers  argue  that  traditional  instructor-centered 
approaches  must  be  replaced  with  more  active  instruction  (Duffy  &  Jonassen,  1991).  Instead  of 
viewing  knowledge  as  an  arbitrary  set  of  facts,  knowledge  needs  to  be  constructed  by  the  learner 
so  that  it  can  be  used  as  a  tool  for  future  learning  activities.  The  focus  of  education  and  training 
needs  to  shift  from  passive  reception  of  data  to  student  knowledge  transformation  wherein  an 
individual  constructs  new  knowledge  through  interactions  and  negotiations.  Constructivist 
principles  include  building  on  student  prior  knowledge,  making  learning  relevant  and 
meaningful,  giving  students  choice  and  autonomy,  and  having  instructors  act  as  co-learners. 
Instructors  might  design  tasks  wherein  learners  solve  real  world  problems,  reflect  on  skills  used 
to  manage  one’s  own  learning,  address  misconceptions  in  their  thinking,  categorize  problems 
around  themes  and  concepts,  and  generally  take  ownership  for  their  own  learning  (Duffy  & 
Cunningham,  1996). 
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There  are  at  least  two  important  variations  of  constructivism-cognitive  constructivist  and 
social  constructivist.  Cognitive  constructivists  tend  to  focus  on  the  individual  construction  of 
knowledge  discovered  or  built  in  interaction  with  the  surrounding  environment.  From  this  point 
of  view,  it  is  important  for  trainers  to  foster  active  learning  environments  wherein  learners 
individually  build  and  construct  new  knowledge.  Basically,  the  cognitive  constructivistic  view 
regards  knowledge  as  internally  represented  in  the  mind  of  the  learner.  Unfortunately,  individual 
notions  of  constructivism  often  fail  to  emphasize  the  vital  social  aspects  of  learning  and 
cognition— the  collaboration,  negotiation,  dialogue,  and  questioning  of  active  learning 
environments.  In  contrast,  social  constructivists  view  learning  as  connection  with  and 
appropriation  from  a  larger  social  context.  Instructional  methods  from  this  latter  view  focus  on 
dialogue,  instructor  co-leaming,  and  the  joint  construction  of  knowledge. 

As  indicated  above,  cognitive  constructivists  focus  on  making  learning  more  relevant, 
building  on  student  prior  knowledge,  and  addressing  misconceptions.  Social  constructivists 
emphasize  human  dialogue,  interaction,  negotiation,  and  collaboration.  Across  both  viewpoints, 
constructivistic  practices  emphasize  active,  generative  learning  wherein  instructors  continue  to 
perform  a  critical  learning  function  as  learning  guides.  The  focus  here  is  on  assisting  learning, 
not  in  directing  or  assessing  it.  The  impact  of  using  guided  or  assisted  learning,  instead  of  either 
mechanistic  or  discovery  learning  systems,  fosters  positive  effects  on  learning.  In  an  e-leaming 
environment  such  assistance  might  include  questioning,  task  structuring,  coaching,  modeling, 
pushing  students  to  articulate  ideas  and  explore  new  avenues,  and  occasional  and  timely  direct 
instruction.  From  a  social  constructivist  point  of  view,  new  learning  communities  can  emerge 
(Scardamalia  &  Bereiter  1996).  For  the  Army’s  transformation  from  the  current  model  of 
classroom  instruction  to  the  visionary  future,  the  implications  are  enormous  for  students, 
instructors,  and  training  managers. 


Learner-Centered  Principles 

During  the  early  1990s,  the  American  Psychological  Association  (APA)  announced  a  set 
of  14  Leamer-C entered  Psychological  Principles  (LCPs)  shown  in  Table  1.  These  were  based  on 
research  from  the  fields  of  learning  and  instruction,  motivation,  and  development  since  the 
emergence  of  cognitive  psychology  in  the  1970s  and  1980s  (Alexander  &  Murphy,  1994;  APA, 
1993;  Learner-centered  psychological  principles  revised,  1996)  (see  Table  1).  Intended  to  help 
with  school  reform  and  redesign,  the  LCPs  address  areas  such  as  fostering  curiosity  and  intrinsic 
motivation,  linking  new  information  to  old  in  meaningful  ways,  providing  learner  choice  and 
personal  control,  nurturing  social  interaction  and  interpersonal  relations,  promoting  thinking  and 
reasoning  strategies,  constructing  meaning  from  information  and  experience,  and  taking  into 
account  learner  social  and  cultural  background.  These  14  principles  have  significant  promise  for 
Web-based  instruction  (Bonk,  Appelman,  &  Hay,  1996;  Bonk  &  Reynolds,  1997).  In  fact,  Bonk 
and  Cummings  (1998)  document  a  dozen  recommendations  for  designing  Web-based  instruction 
from  a  leamer-centered  perspective.  These  guidelines  describe  the  need  for  creating  a 
psychologically  safe  environment,  facilitating  learning,  electronic  mentoring,  and  other  related 
ideas.  In  a  nutshell,  the  LCPs  provide  the  backdrop  for  thinking  about  online  instruction. 
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Table  1 

Learner-Centered  Psychological  Principles  Revised  (see  Appendix  A  for  detailed  table) 


Cognitive  and  Metacognitive  Factors 

1 .  Nature  of  the  learning  process. 

2.  Goals  of  the  learning  process. 

3.  Construction  of  knowledge. 

4.  Strategic  thinking. 

5.  Thinking  about  thinking. 

6.  Context  of  learning. 


Motivational  and  Affective  Factors 


7.  Motivational  and  emotional  influences  on  learning. 

8.  Intrinsic  motivation  to  learn. 

9.  Effects  of  motivation  on  effort. 


Developmental  and  Social  Factors 

10. 

Developmental  influences  on  learning. 

11. 

Social  influences  on  learning. 

Individual  Differences 

12. 

Individual  differences  in  learning. 

13. 

Learning  and  diversity. 

14. 

Standards  and  assessment. 

For  a  full  text  of  the  principles  listed  as  well  as  additional  rationale  and  explanation,  refer  to  the  APA  Website 
(Appendix  B)  or  write  to  the  APA  for  the  December,  1995  report  "The  Learner-Centered  Psychological  Principles: 
A  Framework  for  School  Redesign  and  Reform".  Permission  to  reproduce  this  list  has  been  granted  by  APA. 


Many  educational  technologists  are  advocating  the  need  to  shift  from  instructor-centered 
to  student-centered  approaches  (Bracewell,  Breuleux,  Laferriere,  Benoit,  &  Abdous,  1998; 
Hannafin  &  Land,  1997;  Harasim,  1990).  Learner-centered  pedagogy  asks  what  students  need  to 
learn,  what  their  learning  preferences  are,  and  what  is  meaningful  to  them.  Web-based 
instruction  provides  opportunities  for  learning  materials,  tasks,  and  activities  to  fit  individual 
learning  styles  and  preferences.  Networks  of  learning  information,  such  as  digital  libraries,  are 
available  to  peak  student  interests  and  ideas.  Such  environments  also  provide  access  to  more 
authentic  learning  communities  than  typically  found  in  conventional  educational  environments. 

In  accordance  with  the  learner-centered  movement,  online  tools  should  provide 
opportunities  to  construct  knowledge,  actively  share  and  seek  information,  generate  a  diverse 
array  of  ideas,  appreciate  multiple  perspectives,  take  ownership  in  the  learning  process,  engage 
in  social  interaction  and  dialogue,  develop  multiple  modes  of  representation,  and  become  more 
self-aware  (Chong,  1998;  Harasim,  1990;  Oliver  &  McLoughlin,  1999).  Simply  stated, 
technology  rich  environments  can  support  learner  engagement  in  meaningful  contexts,  thereby 
increasing  ownership  over  their  own  learning  (Chung,  Rodes,  &  Knapczyk,  1998).  A  detailed 
look  at  the  examples,  functions,  and  supporting  research  for  learner-centered  environments  can 
be  found  in  Hannafin  and  Land  (1997). 
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Doherty  (1998)  noted  that  that  emergence  of  hypermedia  technology  combined  with 
asynchronous  learning  networks  provides  greater  opportunity  for  learners  to  take  control.  She 
argues  that  learner  control  will  be  the  most  dominant  characteristic  of  this  new  form  of 
instruction.  Clearly,  the  “learn  anytime,  anywhere,”  manner  of  thinking  will  foster  additional 
expectations  for  greater  learner  control  and  learner  options.  With  increasing  expectations  that  a 
learner  will  be  guiding  his  or  her  own  learning,  instructors  need  to  develop  pedagogical 
strategies  and  employ  technological  tools  that  foster  self-directed  student  inquiry  and 
investigation.  In  such  an  environment,  tools  and  tactics  for  student  manipulation  of  information, 
discovery,  generation  of  artifacts,  and  sharing  of  knowledge  are  highlighted  (Hannafin  &  Land, 
1997).  When  this  occurs,  students  can  examine  problems  at  multiple  levels  of  complexity, 
thereby  deepening  understanding. 

As  an  example,  Hannafin,  Hill,  and  Land  (1997)  focus  on  the  student-centered  benefits  of 
open-ended  learning  environments  (OELEs).  In  OELEs,  knowledge  evolves  as  understanding  is 
modified  and  tested,  while  learners  begin  to  evaluate  their  own  learning  needs.  Basically,  the 
focus  is  on  relevant  and  meaningful  problems  linked  to  everyday  experiences.  In  accordance 
with  the  vision  of  student-centered  environments,  OELEs  support  self-regulated  learning,  enable 
novices  to  negotiate  through  complex  problems,  showcase  knowledge  interrelationships,  anchor 
concepts  in  real  world  events,  and  nurture  various  problem  solving  processes.  Clearly,  these  are 
complex  but  powerful  learning  environments. 

Advances  in  interactive  and  collaborative  technology  is  forcing  instructional  designers 
and  technology  users  to  confront  and  envision  learner-centered  instruction  as  well  as  their  role  in 
it.  Fortunately,  the  Web  is  emerging  as  a  viable  teaching  and  learning  platform  for  learner- 
centered  instruction  at  the  same  time  that  there  is  a  call  for  incorporating  learner-centered 
approaches  in  education.  It  is  difficult  to  tell  whether  this  will  lead  to  serendipitous  or  tremulous 
events  or  both.  However,  what  is  clear  is  that  there  is  a  dearth  of  pedagogical  tools  for  Web 
instruction  (Bonk  &  Dennen,  1999;  McLoughlin  &  Oliver,  1999).  Most  Web  tools  available 
today  do  not  help  transform  education  as  promised.  Instead,  e-leaming  courseware  simply 
facilitates  course  administration  and  registration  procedures. 

Web-Based  Learning  Models 

Most  Web  courseware  is  embedded  with  devices  for  tracking,  managing,  and  controlling 
student  learning,  rather  than  innovative  ways  to  nurture  student  control  and  responsibility  for 
learning.  This  situation  is  not  really  surprising  since  such  courseware  tools  emanate  from  a 
behavioral  learning  model  (Firdyiwek,  1999).  At  the  same  time,  there  is  an  outbreak  of  ideas 
related  to  creative  approaches  for  teaching  on  the  Web  (Bonk  &  Reynolds,  1997).  Levin  and 
Waugh  (1998)  detail  approaches  such  as  online  collaborative  teaming,  online  questioning  and 
answering,  Web  resource  searching  and  evaluation,  project  generation  and  coordination,  and 
student  publication  of  work.  Moreover,  McLoughlin  and  Oliver  (1999)  argue  for  the 
development  of  tools  for  parallel  problem  solving,  simulating  course  material,  information 
exchange,  database  creation,  and  case-based  projects.  Wood  (1999)  points  to  new  instructional 
opportunities  to  locate  information  (i.e.,  scavenger  hunts),  conduct  research,  analyze  data,  take 
part  in  virtual  tours,  exchange  and  publish  information,  and  solve  problems.  The  possibilities 
exist,  therefore,  for  rich  electronic  learning,  but,  for  the  most  part,  pedagogically  sound  and 
exciting  Web  courseware  tools  have  yet  to  be  developed  to  take  advantage  of  such  opportunities. 


9 


Some  might  argue  that  educators  simply  do  not  know  how  to  utilize  emerging  Web 
technologies.  In  effect,  the  pace  of  change  is  so  rapid  that  pedagogical  models  are  needed  to 
help  create  and  understand  Web  tools  from  a  constructivist  or  learner-centered  perspective 
(Bracewell  et  al.,  1998).  As  Salomon  (1998)  has  noted,  for  the  first  time  in  history,  technologies 
are  outpacing  pedagogical  and  psychological  rationale.  While  tools  might  afford  new  forms  of 
inquiry  and  project-based  learning,  there  is  a  need  for  task  structuring  to  guide  knowledge 
exploration  and  communication  among  learning  participants  (Bracewell  et  al.,  1998).  Bourne 
(1998),  for  instance,  provides  a  model  of  potential  shifts  in  faculty  instructional  roles  with  more 
time  projected  for  mentoring  and  less  for  testing. 

Other  models  look  at  the  degree  to  which  the  Web  is  embedded  or  integrated  into  a 
course  (Bonk,  Cummings,  Hara,  Fischler,  &  Lee,  2000;  Mason,  1998)  as  well  as  the  forms  and 
directions  of  interaction  utilized  by  Web  courses  (Cummings,  Bonk,  &  Jacobs,  2000).  Bonk 
(1998)  responded  to  the  lack  of  sound  tools  by  creating  interactive  tools  for  online  portfolio 
feedback,  profile  commenting,  and  Web  link  rating,  while  Oliver  and  McLoughlin  (1999)  are 
building  tools  for  online  debates,  reflection,  concept  mapping,  and  student  surveying  and 
discussion.  Given  these  recent  trends,  the  coming  decade  should  witness  a  growth  spurt  in 
pedagogically-based  e-leaming  technologies. 

Trends  in  pedagogy  are  converging  with  the  emergence  of  e-leaming  technologies  that 
allow  for  greater  learner  control,  personal  responsibility,  and  collaboration.  Self-directed 
learners  who  want  meaningful  and  engaging  activities  as  well  as  instructors  willing  to 
experiment  with  a  variety  of  techniques  and  practices  to  individualize  learning  tend  to  be  more 
attracted  to  e-leaming  settings  (National  Center  for  Education  Statistics,  1999;  Wagner  & 
McCombs,  1995).  Fortunately,  e-leaming  is  a  unique  context  wherein  learner-centered 
principles  are  particularly  relevant  as  students  become  the  center  of  the  learning  environment.  In 
fact,  in  successful  online  courses,  students  might  assume  significant  instructional  roles  such  as 
offering  instructional  tips  and  constructing  new  knowledge  that  were  once  the  domain  of  the 
instructor  (Harasim,  1993).  Along  these  same  lines,  Levin  and  Ben-Jacob  (1998)  predict  that  a 
key  component  of  learning  in  higher  education  at  the  start  of  this  millennium  will  be 
collaborative  learning.  Such  student-centered  learning  environments  will  undoubtedly  be 
flavored  with  team  learning  opportunities. 

Learning  Team  Centered  Approach 

In  training  environments,  both  IBM  and  the  Lotus  Institute  have  published  white  papers 
that  address  the  need  to  extend  learner-centered  approaches  to  “learning  team  centered” 
approaches  (Kulp,  1999;  Lotus  Institute,  1996).  According  to  these  reports,  not  only  do  e- 
leaming  environments  offer  opportunities  for  actively  interpreting,  questioning,  challenging, 
testing,  and  discussing  ideas,  but  they  provide  a  means  to  collaboratively  create  and  share  new 
knowledge. 

The  role  of  the  instructor  in  such  an  environment  is  to  facilitate  student  information 
generation  and  sharing,  not  to  control  the  delivery  and  pace  of  it.  A  key  goal  of  team-based 
learning  activities  is  to  apply  expertise  and  experience  of  the  participants  to  a  group  problem 
solving  situation  or  research  project  that  helps  participants  accomplish  something  that  they  could 
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not  achieve  individually.  Other  objectives  include  the  fostering  of  teamwork,  communication, 
and  listening  skills  (Lotus  Institute,  1996).  While  Kulp  (1999)  admits  that  small  team 
collaboration  in  e-leaming  requires  significantly  more  time  and  effort,  it  can  generate  new 
knowledge,  attitudes,  and  behavior.  He  recommends  the  use  of  roles  such  as  coordinator/leader, 
starter  or  resource  investigator,  summarizer,  scribe,  encourager,  specialist,  implementer,  and 
checker.  The  following  section  will  shed  some  light  on  how  an  instructor  effectively  assumes 
such  roles. 


New  Role  for  Instructors  Online 

If  the  Army  is  to  gain  the  full  benefits  of  online  instruction,  a  significant  change  in  the 
preparation  of  instructors  will  be  required.  The  lessons  from  education  are  that  online  learning  is 
an  entirely  new  type  of  educational  experience  requiring  a  redesign  of  instructor  roles, 
responsibilities,  and  commitments  (Besser  &  Bonn,  1997)  as  well  as  support  and  training  for 
those  teaching  online  (Lawrence,  1996-97).  The  potential  instructional  roles  might  seem 
daunting.  As  student-centered  activities  are  increasingly  facilitated  by  emerging  technology,  the 
role  of  the  faculty  member  or  instructor  shifts  to  facilitator,  coach,  or  mentor  who  provides 
leadership  and  wisdom  in  guiding  student  learning  (Dillon  &  Walsh,  1992;  Doherty,  1998).  Of 
course,  until  instructors  feel  comfortable  and  gain  experience  in  this  new  role,  online  courses 
may  experience  higher  than  expected  drop  out  rates  (Carr,  2000a;  Appendix  B). 

Instructors  have  a  number  of  roles  that  they  can  assume  online  such  as  chair,  host, 
lecturer,  tutor,  facilitator,  mediator,  mentor,  provocateur,  observer,  participant,  co-leamer, 
assistant,  and  community  organizer.  From  one  perspective,  a  good  moderator  is  like  a  successful 
host  or  hostess:  he  or  she  must  know  how  to  connect  guests  together  at  the  party  with  similar 
interests  and  bring  those  hiding  on  the  fringes  into  the  community  (Rogan  &  Denton,  1996). 
From  another  perspective,  it  might  be  important  for  the  instructor  to  act  as  a  co-leamer  or 
participant  in  online  activities.  Rice-Lively  (1994)  found  that  the  online  instructor  must  be 
flexible  in  constantly  shifting  between  instructor,  facilitator,  and  consultant  roles.  At  the  same 
time,  instructors  must  create  an  ethos  of  mutual  support  and  community  so  that  students  with 
limited  technology  experience  can  perform  well  in  these  environments  (Ross,  1996).  This  is  not 
particularly  easy.  Fortunately,  many  of  the  possible  instructor  roles  can  be  assigned  to  students, 
outside  experts,  or  teaching  assistants  (Paulsen,  1995a;  Selinger,  1999). 

Guidelines  for  Facilitation 

Researchers  have  suggested  various  guidelines  for  online  facilitation.  Cummings  (2000) 
offers  a  sequence  of  steps  for  use  within  a  virtual  debate  (see  Table  2).  These  steps  provide  a 
template  for  fostering  social  interaction  online.  Mason  (1991)  advocates  the  organizational, 
social,  and  intellectual  roles  of  the  online  instructor.  The  organizational  role  entails  setting  the 
agenda,  objectives,  timetable,  and  procedural  rules  for  posting  and  interaction. 

Recommendations  for  instructors  include  patience,  varying  the  participation,  avoiding 
lecturing,  inviting  guest  speakers,  spurring  discussion,  and  addressing  unanticipated  activities  or 
problems.  The  social  role  involves  sending  welcoming  messages,  thank  you  notices,  feedback 
on  student  inputs,  and  a  generally  friendly,  positive,  and  responsive  tone.  Instructional  caveats 
concerning  the  social  role  include  reinforcing  good  discussion  behaviors  and  inviting  students  to 
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be  candid  about  the  way  the  course  is  proceeding.  Of  the  three  roles  Mason  (1991)  describes,  the 
intellectual  role  is  the  most  crucial  since  it  includes  probing  responses,  asking  questions, 
refocusing  discussion,  setting  goals,  explaining  tasks  and  overlooked  information,  weaving 
disparate  comments,  synthesizing  key  points  raised  and  identifying  unifying  themes,  directing 
discussion,  and  generally  setting  and  raising  the  intellectual  climate  of  the  course  or  seminar. 
Knowing  when  to  summarize,  when  to  expose  conflicting  opinions,  and  when  to  request 
comments  on  specific  issues  is  also  critical  for  this  role. 

Table  2 

Sequence  of  steps  within  a  virtual  debate  (Cummings,  2000) _ _ 

Virtual  Debate  Steps 

’  1 .  Instructor  selects  controversial  topic  with  input  from  class. 

2.  Instructor  divides  class  into  subtopic  pairs. 

3.  Instructor  assigns  subtopic  pairs. 

4.  Critics  and  defenders  post  initial  positions. 

5.  Students  review  all  initial  position  statements. 

6.  Students  reply  to  at  least  two  position  statements  with  comments  or  questions. 

7.  Each  student  rebuts  opposing  initial  statement  or  individual  in  his/her  pair. 

8.  Based  on  a  review  of  all  statements,  comments,  and  questions,  students  formulate 
personal  positions. 

9.  Students  post  personal  position  statements  in  private  forums. _ 


A  report  from  a  year-long  faculty  seminar  on  online  teaching  and  learning  at  the 
University  of  Illinois  recommended  that,  in  attempting  to  facilitate  online  collaborative  learning, 
instructors  be  patient,  flexible,  responsive,  and  clear  about  expectations  and  norms  for 
participation.  In  addition,  such  process  facilitation  requires  that  they  limit  lecturing,  while 
monitoring  and  prompting  student  participation,  organizing  student  interactions,  and  writing 
integrative  or  weaving  comments  on  occasion.  Furthermore,  they  need  to  find  ways  for 
individuals  and  small  groups  to  assume  teaching-related  roles  from  time-to-time  (The  Report  of 
the  University  of  Illinois  Teaching  at  an  Internet  Distance  Seminar,  1999).  Instructors  also  need 
to  become  adept  at  promoting  interaction,  addressing  multiple  learning  styles,  performing  needs 
assessments,  and  projecting  a  friendly  image  (Thach,  1993). 

Instructor  actions.  E-leaming  instructor  presence  is  sensed  by  students  through  messages 
posted  to  the  online  conference.  Ashton,  Roberts,  and  Teles  (1999)  suggest  that  categorizing  the 
online  acts  of  instructors  into  four  categories — pedagogical,  managerial,  technical,  and  social — 
might  be  helpful  in  understanding  the  role  of  the  instructor  in  collaborative  online  environments. 
Pedagogical  action  includes  feedback,  providing  instructions,  giving  information,  offering  advice 
and  preferences,  summarizing  or  weaving  student  comments,  and  referring  to  outside  resources 
and  experts  in  the  field.  In  effect,  the  pedagogical  role  relates  to  direct  instructor  involvement  in 
class  activities.  Similarly,  online  managerial  actions  involve  overseeing  task  and  course 
structuring.  Managerial  actions  include  coordinating  assignments,  discussions,  and  the  course. 
Technical  actions  relate  to  helping  with  user  or  system  technology  issues.  Finally,  social  actions 
might  include  instructor  empathy,  interpersonal  outreach  (e.g.,  welcoming  statements, 
invitations,  and  apologies),  discussion  of  one’s  own  online  experiences,  and  humor.  Ashton  et 
al.  (1999)  suggest  that  future  research  look  at  the  role  of  the  instructor  from  the  start  to  end  of  an 
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online  course,  across  instructors,  across  offerings  of  the  same  course,  and  across  different 
courses.  Additionally,  one  might  wish  to  explore  how  different  technologies  and  pedagogical 
strategies  change  the  instructional  interaction  patterns  and  help  promote  community  building. 
Such  issues  are  addressed  in  later  sections  of  this  manuscript. 

Both  Bonk  and  Cummings  (1998)  and  Bailey  and  Luetkehans  (1998)  provide  several  tips 
for  online  instructors  to  create  learner-centered  environments.  These  include  the  need  to  develop 
psychologically  safe  learning  environments  where  there  is  extensive  student  interaction  and 
elaboration.  Reduction  in  cyber-stress  must  be  one  of  the  key  initial  goals.  To  accomplish  this, 
one’s  expectations  must  be  clear  and  prompt.  In  addition,  online  learning  teams  must  be 
assembled  according  to  the  tasks  involved  and  available  talent  and  interests.  Students  in  small 
groups  must  have  open-ended  problems  with  some  degree  of  choice,  though  timely  instructional 
support  is  needed  when  struggling  to  reach  consensus  and  prompt  feedback  on  decisions. 
Moreover,  student  assignments  should  build  on  their  experience  and  prior  knowledge,  while 
allowing  students  to  find  ways  to  utilize  the  Web  resources  and  share  them  with  team  members. 
The  instructor,  for  instance,  might  intervene  to  indicate  where  the  group  members  have  found 
some  common  ground. 

Instructors  should  also  take  advantage  of  both  public  and  private  forms  of  feedback  as 
well  as  online  questioning  techniques  and  facilitation  that  can  stimulate  student  reflection. 

Along  these  same  lines,  instructors  should  attempt  to  utilize  the  Web  for  social  interaction  and 
mentoring  to  peers  and  experts  outside  the  class.  Students  can  be  electronically  apprenticed  with 
timely  insights  and  suggestions  from  other  instructors  and  students  located  elsewhere.  Finally,  in 
student-centered  e-leaming  environments,  instructors  should  create  an  atmosphere  wherein 
participants  avoid  quick  judgment  and  overall  negative  criticism  or  personal  attacks  on  one’s 
projects. 

Online  Moderator  Research 

While  guidelines  such  as  those  presented  above  may  be  useful,  what  do  online  instructors 
really  do?  Research  indicates  that  online  instructors  tend  to  rely  on  simple  tools  such  as  e-mail, 
static  or  dynamic  syllabi,  Web  links  to  course  material,  posting  lecture  notes  online,  and 
accepting  student  work  online,  while  significantly  fewer  use  online  chatrooms,  multimedia 
lectures,  online  examinations,  animation,  and  video  streaming  (Peffers  &  Bloom,  1999;  see 
Appendix  B). 

A  study  by  Mclsaac,  Blocher,  Mahes,  and  Vrasidas  (1999)  indicates  that  instructor  time 
is  divided  into  numerous  tasks.  In  their  study,  e-leaming  instructors  allocated  their  time  to 
planning  and  preparation  (10%),  online  teaching  (17%),  administration  (15%),  interaction  with 
peers  (21%),  interaction  with  students  (15%),  and  interaction  with  content  (22%).  Based  on 
these  findings,  Mclsaac  et  al.  contend  that  online  instructors  tend  to  be  more  concerned  about 
encouraging  student  participation  and  the  quality  of  interaction  than  might  be  expected  in 
traditional  settings.  As  a  result  of  this  change  in  role,  Web  courses  take  significant  amounts  of 
time  for  instructors  to  create  and  coordinate  (Gaud,  1999).  To  help  instructors  assume  such  new 
roles,  they  need  assistance  and  advice  both  in  the  development  and  delivery  of  Web  courses 
(Lawrence,  1996-97). 
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While  e-leaming  moderating  talents  are  not  easily  explained  and  understood  and  online 
success  stories  are  not  pervasive,  some  guidelines  are  emerging.  According  to  Selinger  (1997), 
successful  moderation  seems  to  include  student  system  familiarity,  encouraging  introductory 
messages,  clear  guidelines  and  purpose,  online  guests,  and  intermittent  summaries  and 
refocusing  of  discussion.  Moderators  should  identify  their  preferred  pedagogical  styles  and 
preferred  forms  of  online  facilitation  and  moderation  (Paulsen,  1995a;  Appendix  B).  As  these 
roles  evolve,  researchers  need  to  ask  just  how  much  student  autonomy  and  interaction  will  they 
promote.  Researchers  might  also  inquire  about  how  often  will  they  intervene  to  offer  advice  and 
feedback  in  student  discussions  and  activities.  Some  initial  research  has  begun  to  answer  these 
questions. 

Significant  teaching  and  learning  changes  are  underway  due  to  the  emergence  and 
popularity  of  e-leaming.  For  instance,  e-leaming  supports  a  more  social  constructivist  learning 
environment  wherein  students  negotiate  meaning  and  are  involved  in  extensive  dialogue  and 
interaction.  The  role  of  the  instructor,  therefore,  is  in  transition  from  director  to  facilitator  or 
moderator  of  learning  (Selinger,  1999).  In  addition,  electronic  learners  are  more  autonomous 
and  independent  in  their  own  learning  than  their  counterparts  in  traditional  classrooms.  E- 
leamers  also  have  greater  opportunities  for  interacting  with  other  learners,  their  instructor,  and 
outside  experts.  Teaching  and  learning  takes  on  a  more  collaborative  feel  in  an  e-leaming 
environment.  Whether  this  is  equally  true  in  education  and  training  environments  is  uncertain. 
What  is  known  is  that  learners  in  both  environments  will  have  greater  opportunities  for  reflection 
and  exploration,  thereby  expanding  ideas  about  when  and  where  learning  in  a  course  actually 
takes  place. 

What’s  next?  Given  the  above  time  barriers  instructors  face,  perhaps  intelligent  agents 
will  be  developed  to  provide  questions  or  advice  on  the  particular  task  students  are  working  on 
(Kearsley,  1993).  Along  these  same  lines,  intelligent  tutoring  systems  or  modules  might  be 
inserted  into  existing  online  courseware  or  specific  tools  to  provide  advice  or  support  for  student 
learning  (Ritter  &  Koedinger,  1996).  Intelligent  tutoring  systems  have  been  successfully  applied 
in  military  settings  (Psotka,  1988).  Such  systems  interpret  or  identify  what  the  student  needs  to 
know  and  suggest  activities,  information,  or  advice.  Given  the  increase  in  distance  education 
and  the  unfamiliarity  of  instructors  in  this  environment,  it  is  certainly  a  ripe  area  for  intellectual 
support  tools  and  features.  As  such  development  occurs,  we  may  begin  to  better  understand  how 
to  successfully  moderate  or  facilitate  e-leaming  environments.  We  also  might  understand  the 
overall  importance  of  research  on  Web-based  instruction. 

Online  Collaboration 

Owston  (1997)  asks  three  basic  questions  in  terms  of  Web-based  instruction:  (1)  Does  it 
increase  learning  access?;  (2)  Can  it  improve  learning?;  (3)  Can  increased  access  and  improved 
learning  be  attained  without  additional  costs?  Unfortunately,  answers  to  Owston’ s  questions  are 
not  readily  available.  In  fact,  most  research  on  whether  the  Web  improves  student  academic 
performance  is  strictly  anecdotal  or  attitudinal  (Usip  &  Bee,  1998)  or  finds  no  significant 
differences.  Too  few  educators  are  asking  critical  questions  about  e-leaming  and,  instead,  rely 
on  such  anecdotal  evidence  or  nothing  at  all  (Windschitl,  1998). 
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Questions  need  to  be  raised  in  both  education  and  training  environments  about  the 
processes  that  occur  during  online  searching,  the  types  of  guidance  needed,  how  online  learners 
reach  common  ground,  and  the  role  of  the  instructor  in  online  collaboration.  Is  there  evidence 
that  students  and  instructors  benefit  from  e-leaming?  Do  these  tools  improve  teaching  and 
learning?  Do  different  types  of  learners  (e.g.,  visual,  verbal)  or  learning  strategies  foster  greater 
achievement  (Kerka,  1998;  Shih,  Ingebritsen,  Pleasants,  Flickinger,  &  Brown,  1998)?  How  do 
e-leaming  training  environments  such  as  in  the  military  and  corporate  sector  differ  from  or 
confirm  results  found  in  higher  education? 

In  addition  to  the  limitations  of  anecdotal  data,  some  argue  that  most  e-leaming  research 
is  flawed  by  the  lack  of  control  groups,  nonrandom  assignment  to  groups,  questionable  validity, 
and  limited  scope  (The  Report  of  the  University  of  Illinois  Teaching  at  an  Internet  Distance 
Seminar,  1999;  Wisher  &  Champagne,  2000).  Of  course,  there  is  much  to  be  learned  from  small 
sample  ethnographic  or  case  study  research.  In  both  quantitative  and  qualitative  research, 
however,  the  research  on  e-leaming  tends  to  focus  on  the  impact  of  individual,  not  multiple 
technologies.  Prior  research,  moreover,  generally  fails  to  consider  student  learning  styles  and 
other  individual  differences  (e.g.,  self-efficacy,  motivation,  and  gender)  in  using  those 
technologies.  Worse  still,  this  research  too  often  lacks  reliable  and  valid  testing  instruments 
while  neglecting  to  theoretically  ground  the  study.  Finally,  it  fails  to  account  for  the  higher  drop 
out  rates  experienced  in  Web-based  instruction  (National  Center  for  Education  Statistics,  1999, 
Phipps  &  Merisotos,  1999;  Wisher  et  al.,  1999). 


Research  Methodologies  in  Collaborative  Environments 

This  section  describes  the  methods  used  by  researchers  to  quantify,  analyze,  and  report 
information  and  data  gathered  during  episodes  of  collaborative  learning.  Collaborative 
environments  support  learners  in  achieving  common  learning  goals.  The  practice  of  using  small 
group  instruction  in  so  many  Army  training  programs  creates  a  natural  candidate  for 
collaborative  environments,  enabling  soldiers  to  draw  from  the  field  experiences  and  multiple 
perspectives  of  other  soldiers  “seated”  online  rather  than  around  the  table.  Covered  below  are 
the  use  of  quantitative  and  qualitative  measurement  instruments,  techniques  to  analyze  content, 
assessment  of  messages,  and  methods  used  for  examining  conferencing  tools,  interaction 
schemes,  critical  thinking,  and  learning  styles. 

Some  scholars  argue  that  standard  experimental  designs  are  not,  practical,  relevant,  or 
ethical  in  online  environments  (Hiltz,  1 990).  It  is  difficult  to  randomly  assign  students  to 
traditional  course  sections  when  they  desire  or  need  the  online  version.  And  it  may  be  unethical 
to  change  the  meeting  times  for  students  relying  on  a  course  to  complete  a  major.  Furthermore, 
students  selecting  online  courses  may  differ  significantly  in  their  maturity,  expectations,  and 
motivation  than  students  in  the  traditional  class.  Scholars  like  Stephen  Ehrmann  (2000)  from  the 
Flashlight  Project  argue  that  notions  of  “normal”  or  conventional  classroom  are  misguided. 
Every  classroom  and  program  has  unique  materials,  methods,  motives,  and  students.  From  this 
perspective,  an  accounting  of  comparable  costs  is  problematic,  especially  given  the  lack  of 
rationale  for  enumerating  costs  per  student  or  class  in  traditional  education. 
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The  literature  on  online  learning  details  both  quantitative  and  qualitative  research 
instruments  (Hiltz,  1990;  Howell-Richardson  &  Mellar,  1996;  Riel  &  Harasim,  1994).  On  the 
quantitative  side,  researchers  often  discuss  usage  patterns,  computer  log  data  (e.g.,  number  of 
participants,  message  number  and  length,  reading  time,  message  creation  time,  etc.),  data  mining, 
video  screen  grabs,  participation  rates,  participation  timing,  free  or  cued  recognition  tasks, 
student  and  instructor  attitudes,  writing  skill  improvement,  peer  responsiveness  and  interactivity, 
and  various  questionnaire  data  (e.g.,  Mowrer,  1996).  Data  mining  tools  now  enable  researchers 
to  quickly  obtain  basic  or  summary  statistics  (number  of  logins,  peak  hours  of  usage,  location  of 
user,  length  of  session,  paths  taken,  messages  by  day  or  week,  etc.),  classification  and  association 
analyses  (e.g.,  grouping  user  by  navigation  types  or  characteristics),  time-series  analyses  (i.e., 
grouping  data  for  time  related  trends  and  similarities),  and  data  visualization  (i.e.,  graphically 
plotting  data  about  usage,  participation,  etc.)  (Harasim,  1999).  Such  tools  can  elucidate  the 
timing  and  quantity  of  student  online  work. 

In  addition  to  computer  log  data,  quantitative  measures  can  also  assess  student  skills  or 
traits.  For  instance,  with  the  heavy  emphasis  on  writing  and  communicating  in  most  online 
learning  environments,  it  is  not  surprising  that  there  is  interest  in  writing  skill  development  and 
changes  in  audience  awareness  or  perspective  taking  (Bonk  &  Sugar,  1998;  Cohen  &  Riel, 

1989).  Other  measures  such  as  student  self-efficacy,  course  satisfaction,  perceived  level  of 

learning,  and  computer  anxiety  in  these  environments  might  also  be  assessed. 

♦ 

Content  Analysis 

The  tools  for  assessment  on  the  qualitative  side  are  also  rich  and  varied.  Here, 
researchers  often  point  to  interaction  and  content  analyses,  discourse  quality,  verbal  protocols, 
message  flow  analysis,  message  thread  analysis,  task  phase  analysis,  semantic  trace  analysis,  the 
classification  of  participant  types,  forms  of  feedback,  reflective  interviews,  observation  logs, 
focus  groups,  retrospective  analyses,  and  user  think  alouds  (e.g.,  Levin,  Kim,  &  Riel,  1990;  Rice- 
Lively,  1994).  In  fact,  so  many  methods  are  mentioned  in  the  literature,  it  is  difficult  to  know 
when  and  where  to  use  them.  Message  thread  analysis  entails  grouping  messages  related  to  one 
another  into  common  message  threads  for  analysis  (Riel  &  Harasim,  1994).  In  contrast,  task 
phase  analysis  is  often  used  to  examine  interaction  patterns  and  activity  during  different 
components  of  a  large-scale  project  or  extended  task  or  unit.  Another  qualitative  technique, 
semantic  trace  analysis,  is  designed  to  map  out  the  development  of  a  single  idea  or  set  of  ideas 
over  time.  Using  this  latter  method,  one  might  discover  that  pivotal  student  contributions 
actually  originated  in  other  classrooms  or  in  discussions  of  e-leaming  affiliates  (Riel  &  Harasim, 
1994).  Focus  groups  and  the  nominal  group  technique  might  foster  discussion  related  to 
sensitive  topics  and  open  issues  that  might  never  be  considered  otherwise  (Eastmond,  1994).  Of 
course,  each  of  these  methods  has  associated  pros  and  cons. 

Messsages.  Levin  et  al.  (1990)  point  out  that  how  often  a  message  is  referenced  by  other 
messages  is  an  indicator  of  the  importance  of  certain  network  participants  and  the  direction  of 
the  online  conversation.  Graphic  displays  of  message  interaction  might  signify  not  only  what 
topics  were  popular  but  also  member  status  and  dominance.  While  this  is  the  first  step  in 
Semantic  Trace  Analysis,  the  next  step  might  be  to  perform  Message  Act  Analysis  wherein  each 
message  within  a  discussion  thread  is  classified  according  to  whether  it  is  in  initiation  of  a 
discussion,  reply,  or  evaluation.  Noting  who  is  performing  such  acts — teacher  or  student — is 
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useful  in  determining  whether  the  online  discussion  is  following  traditional  teacher  domination 
patterns  (i.e.,  the  quintessential  IRE  sequences)  or  allowing  for  more  student-centered  learning. 
Interestingly,  Levin  et  al.  found  substantial  differences  between  instructional  patterns  in  face-to- 
face  (FTF)  classes  and  online  networks,  favoring  online  classes. 

These  same  researchers  recommend  Message  Flow  Analysis  for  analyzing  the  density  of 
messages  in  units  of  time  (e.g.,  weeks  of  a  semester,  days  of  the  week,  hours  of  the  day, 

(Harasim,  1999)).  Their  experience  in  graphing  such  messages  is  that  there  are  peaks  and 
valleys  in  electronic  networks.  The  troughs  might  occur  at  the  start  of  a  semester  when  training 
is  taking  place  as  well  as  during  exams  and  holidays.  Message  Flow  Analysis  can  indicate  when 
messages  peak  early  and  then  fall  off  (see  Kirkley,  Savery,  &  Grabner-Hagen,  1998),  as  well  as 
when  events  start  out  more  slowly. 

There  certainly  is  no  lack  of  e-leaming  data  in  educational  settings.  Given  the  wealth  of 
quantitative  and  qualitative  methods,  it  is  not  too  surprising  that  there  are  many  evaluation 
frameworks.  In  weeding  through  online  data  sets,  Riel  and  Harasim  (1994)  contend  that  research 
can  be  categorized  into  three  areas: 

1 .  the  technical  and  social  structure  of  the  network  environment 

2.  the  social  interaction  among  the  participants  in  the  course,  project,  or  network  activity; 

and 

3.  the  effects  of  the  online  experience  on  the  individuals. 

Within  these  three  research  areas,  data  analysis  can  be  of  a  fine  grain  in  analyzing  all 
events  or  it  can  be  fairly  coarse  or  global  (Rouet  &  Passerault,  1999).  At  the  same  time,  the 
environment  for  this  research  can  range  from  laboratory  studies  of  individual  cognition  to  design 
experiments  of  classrooms  and  small  group  collaboration  to  larger  scale  testbeds  or  even  reform 
consortia  (Gomez,  Fishman,  &  Pea,  1998).  When  appropriately  combined  and  understood,  these 
methods  allow  researchers  to  analyze  social  and  cognitive  activity  as  it  unfolds.  For  instance, 
computer  logging  devices  and  dialogue  transcript  records  provide  researchers  with  useful  tools 
for  tracking  student  development  both  over  extended  periods  of  time  as  well  as  within  a  single 
online  session. 

As  Fetterman  (1998)  accurately  points  out,  technology  tools  are  playing  an  increasing 
role  in  e-leaming  research.  There  are  now  Web  tools  for  data  collection,  analysis,  and  reporting. 
Technologies  exist  for  recording  online  interviews,  sharing  preliminary  and  final  data  and 
resources,  organizing  field  notes,  searching  database  engines,  locating  needed  resources,  and 
working  with  other  team  members  at  a  distance.  Indeed,  one  might  describe  this  as  a  revolution 
in  course  assessment  tools.  Electronic  surveys  are  also  growing  in  popularity  and  usage 
(Champagne,  1998).  While  equipment,  unfamiliarity,  and  misperceptions  about  time  required 
and  complexity  of  the  task  limit  the  response  rates  of  electronic  surveys  (Bertot  &  McClure, 
1996),  familiarity  with  such  tools  will  increase  response  rates. 

As  the  e-leaming  assessment  tools  evolve,  researchers  might  look  at  both  quantitative  and 
qualitative  data  with  student  questionnaires  and  related  evaluations,  performance  measures, 
observations  of  interaction  patterns,  technology  evaluations,  completion  and  attrition  rates,  and 
cost-benefit  analyses  (Owston,  1999;  Phelps,  Wells,  Ashworth,  &  Hahn,  1991;  Wetzel,  1996). 


17 


Collaborative  Tool  Interaction  Research 

Collaborative  learning  has  the  potential  to  foster  the  interaction  and  social  support 
traditionally  lacking  in  distance  learning  environments.  In  response,  there  has  been  a  wave  of 
research  and  design  taking  place  in  the  field  of  Computer  Supported  Collaborative  Learning 
(CSCL)  and  Computer  Supported  Collaborative  Work  (CSCW).  The  first  CSCW  conference  was 
held  in  1986  in  Austin,  Texas  with  300  people  from  a  variety  of  backgrounds  such  as  artificial 
intelligence,  human-computer  interaction,  office  information  systems,  computer  science, 
anthropology,  and  psychology  (Bannon  &  Hughes,  1993;  Appendix  B).  As  a  field,  CSCW 
employs  groupware  tools  and  group  decision  support  systems  to  increase  the  effectiveness  of 
work  teams  by  facilitating,  augmenting,  and  redefining  their  communication  and  interaction 
activities  (Koschmann,  1994;  Wang  &  Bonk,  2000).  Groupware  are  computer-based 
technologies  that  support  groups  of  people  to  complete  a  common  task  or  perform  a  common 
goal  despite  time  and  space  separation  and  variations  in  team  size  (Davenport  &  McKim,  1995). 
Tools  here  include  e-mail,  computer  conferencing,  workflow  management,  collaborative  writing 
and  co-authoring  software,  document  management,  and  calendaring.  Groupware  products  such 
as  Lotus  Notes,  Timbuktu,  and  GroupSystems  are  used  in  business  to  generate  productivity  gains 
(Kittner  &  Van  Slyke,  1997;  McLellan  &  Knupfer,  1993). 

CSCL  is  the  younger  sibling  of  CSCW  and  is  more  focused  on  how  collaborative 
environments  enhance  student  learning  and  teamwork  in  academic  tasks.  The  field  extends  work 
on  using  technology  as  a  tool  that  augments  and  enhances  learning,  to  thinking  about  how  it  also 
can  be  used  for  student  collaboration,  learning  related  interaction,  and  knowledge  building 
(Koschmann,  1996).  CSCL  tools  might  support  the  communication  and  linking  of  ideas,  the 
structuring  of  group  dialogue,  the  tracking  of  decisions,  the  visualization  and  representation  of 
ideas,  the  generation  and  evaluation  of  ideas,  the  collection  and  analysis  of  data,  and  any  online 
mentoring  and  feedback  (Dede,  1996b).  Increasingly,  the  focus  is  on  supporting  the  creation  of 
knowledge  building  communities  with  extensive  online  discussion,  knowledge  sharing  and 
evaluation,  and  the  storage  of  ideas  in  communal  databases  (Hewitt  &  Scardamalia,  1998). 
Whereas  the  first  CSCL  workshop  took  place  in  1991  (Koschmann,  1994),  the  first  international 
CSCL  conference  was  not  held  until  1995  in  Bloomington,  Indiana.  This  conference  produced 
both  a  popular  book  (Schnase  &  Cunnius,  1995)  and  a  Web  site  (Appendix  B).  Instructional 
designers,  instructional  technologists,  educational  media  specialists,  educational  psychologists, 
learning  theorists,  computer  scientists,  human-computer  interaction,  and  sociologists  populate 
CSCL. 


Scholars  in  these  fields  want  to  help  people  leam  or  work  in  teams  using  technology.  In 
fact,  both  fields,  CSCW  and  CSCL,  are  interested  in  how  to  electronically  share  goals  and 
creations,  the  formation  of  mutual  understandings,  and  the  tools,  tasks,  and  group  sizes  that 
foster  online  social  interaction  and  collaboration.  In  both  environments,  the  sharing  and 
distributing  of  information  is  valued  and  nurtured.  As  Michael  Schrage’s  journeys  across  the 
country  a  decade  ago  indicated,  these  tools  create  a  socially  shared  space  for  users  to  collaborate 
and  exchange  ideas.  Schrage  (1990,  p.  40)  defines  collaboration  as: 

.the  process  of  shared  creation',  two  or  more  individuals  with  complementary  skills 
interacting  to  create  a  shared  understanding  that  none  had  previously  possessed  or  could  have 
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come  to  on  their  own.  Collaboration  creates  a  shared  meaning  about  a  process,  a  product,  or 
an  event. 

As  indicated  by  this  definition,  collaboration  impacts  the  way  in  which  people  share  thoughts 
and  ideas.  While  both  CSCL  and  CSW  shed  light  on  the  gaps  in  leamer-centered  collaborative 
tool  research,  the  CSCL  area  has  more  consistently  addressed  this  area. 

Group  Interactions.  Groups  using  electronic  brainstorming  tools  often  generate  more 
unique  ideas  and  of  a  higher  quality  than  verbal  groups  without  such  support  (Valacich,  Paranka, 
George,  &  Nunamaker,  1 993).  Electronic  tools  have  a  greater  capacity  to  support  concurrent 
discussions  than  the  more  serial  medium  of  verbal  communication.  In  addition,  less  dominant 
individuals  tend  to  participate  more  in  computer-mediated  tasks  than  FTF  tasks  (Citera,  1998). 

These  findings  about  student  dominance  tend  to  hold  in  other  studies.  For  instance,  using 
Interaction  Process  Analysis  (IPA),  a  method  created  in  the  1950s  for  studying  group  interaction, 
Hiltz,  Johnson,  and  Turoff  (1986),  also  found  less  dominance  in  the  computer  condition  than  in 
the  FTF  condition.  However,  the  lack  of  dominance  appeared  to  impair  the  ability  to  reach 
consensus  (see  Table  3  for  12  IPA  categories).  Similarly,  Warschauer  (1997),  in  his  review  of 
the  literature  on  computer-mediated  collaborative  theory,  mentioned  that  group  member  status 
(e.g.,  graduate  or  undergraduate  student)  is  less  important  in  electronic  settings.  In  effect,  in 
electronic  discussion,  there  is  less  time  pressure  and  fewer  social  clues,  nonverbal  cues,  and 
chances  for  intimidation  than  in  FTF  situations.  He  also  found  that  FTF  groups  often  reach 
consensus  by  the  third  speaker;  a  point  wherein  electronic  discussions  are  typically  far  from 
complete.  In  contrast,  Hiltz  et  al.  (1986)  found  that  participants  in  the  FTF  condition  produced 
significantly  more  ideas,  thereby  expediting  consensus.  There  are  many  such  conflicting  and 
inconclusive  findings  in  the  e-leaming  literature. 

Table  3 


Interaction  Process  Analysis  Categories  (Bales,  1950  as  cited  in  Hiltz  et  al.,  1986) 


IPA  Category 

1 .  Show  solidarity 

7.  Asks  for  orientation 

2.  Shows  tension  release,  jokes 

8.  Asks  opinion 

3.  Agrees 

9.  Asks  for  suggestion 

4.  Gives  suggestions 

10.  Disagrees 

5.  Gives  opinions 

1 1 .  Shows  tension 

6.  Gives  orientation 

12.  Shows  antagonism 

Warschauer  (1997)  also  noted  that  CMC  studies  tend  to  find  greater  amounts  of  student 
participation  than  traditional  classrooms  both  as  a  percent  of  total  talk  time  and  in  terms  of 
direction  toward  fellow  students.  In  effect,  students  have  a  voice  instead  of  simply  responding  to 
instructor  prodding.  According  to  his  review,  studies  that  focus  on  computer-mediated 
collaborative  writing  find  that  students  write  more  and  of  a  higher  quality,  are  more 
collaborative,  and  become  more  versatile  writers.  In  effect,  there  is  a  general  movement  from 
teacher-centered  to  student-centered  learning  environments  in  computer-mediated  collaborative 
learning  (see  also  Hara,  Bonk,  &  Angeli,  2000). 


19 


Still  the  instructor  is  key  to  effective  learning.  In  Lau  and  Hayward’s  (1997;  Appendix 
B)  study  of  student  online  surveys,  application  usage  logs,  keystrokes,  focus  groups,  and  help 
desk  usage,  for  instance,  group  members  needed  clear  roles,  realistic  expectations,  and 
significant  resource  and  facilitator  support.  Such  research  findings  also  replicate  studies 
mentioned  earlier — online  instruction  or  facilitation  is  complex  and  time-consuming. 
Furthermore,  action  research  from  Kittner  and  Van  Slyke  (1997)  on  group  decision  support 
systems  in  higher  education  indicates  that  instructors  should  plan  ahead,  have  contingency  plans, 
keep  experience  logs,  and  test  the  system  ahead  of  time. 

Additional  research  is  needed  in  CSCL  and  CSCW  environments,  especially  those  where 
individuals  are  apprenticed  into  an  authentic  learning  environment.  Computer  conferencing  in 
these  environments  is  a  significant  event  since  it  allows  academics,  students,  and  practitioners  to 
work  in  more  close  proximity  (Pearson  &  Selinger,  1999),  while  expanding  the  forms  and 
opportunities  for  interaction  (Cummings  et  al.,  2000).  Research  on  electronic  collaboration 
during  field  placements  of  preservice  teachers,  for  example,  reveal  that  CMC  tends  to  provide 
social  supports  that  acknowledge  and  support  similar  experiences.  However,  these  environments 
fail  to  support  extensive  reflection  on  course  content  (Admiraal,  Lockhorst,  Wubbels,  Korthagen, 
&  Veen,  1997).  Even  CMC  environments  designed  with  pedagogical  structure  and  instructions 
for  connecting  course  knowledge  to  field  experiences  often  results  in  student  story  telling, 
sharing  of  ideas,  opinionated  feedback,  and  social  acknowledgements,  not  deep  course 
connections  and  elaborate  explanations  of  terms  and  principles  (Bonk,  Hansen,  Grabner-Hagen, 
Lazar,  &  Mirabelli,  1998).  Using  learner-centered  principles,  the  Bonk,  Hansen  et  al.  study 
specifically  focused  on  how  online  scaffolding  can  come  from  different  participants — peers, 
instructors,  graduate  students,  conference  moderators,  etc.  They  found  that  students  tended  to 
use  online  networks  for  the  social  interaction  not  for  intense  cognitive  and  metacognitive 
exchanges.  Once  students  move  from  simply  observing  to  becoming  practicing  teachers,  seeking 
help  will  perhaps  become  more  important  than  moral  support,  keeping  in  touch  with  others,  or 
sharing  ideas  (Selinger,  1997).  (Table  4  is  a  modification  of  the  scheme  used  by  Selinger  to 
analyze  student  online  contributions.) 

Table  4 

Uses  of  Online  Conferencing  (adapted^  and  modified r  from  Selinger^  1997) _ 

_ Category  of  Online  Contributions _ 

•  Seeking  Information/Ideas/Help 

•  Offering  Help  or  Support 

•  Clarifying  Issues/Concepts 

•  Moral  Support  (feedback  on  ideas,  sharing  enthusiasm) 

•  Moral  Support  (communicating  or  keeping  in  touch  with  students/peers) 

•  Sharing  Ideas,  News,  and  Techniques  (e.g.,  recent  events,  recent  debates) 

•  Sharing  Resources  Found  or  Created 

•  Contacting  Students 

•  Contacting  Instructor/Moderator  _ 


Conferencing  tools.  The  design  of  the  technology  tools  can  have  an  impact  on  the 
collaboration  in  education  and  training  settings.  Pychyl,  Clarke,  and  Abarbanel  (1999)  found 
that  groups  function  better  when  collaborative  tools  on  the  Web  clearly  demarcate  individual 
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member  contributions,  structure  participation,  provide  easy  access  to  group  member 
contributions  allow  for  individual  workspaces,  and  provide  consistent  technology  for  all 
participants.  Along  these  same  lines,  Duffy,  Dueber,  and  Hawley  (1998)  built  the  Asynchronous 
Collaboration  Tool  (ACT)  to  support  user  critical  thinking  and  collaboration  through 
conversational  and  issue-based  discussions.  Using  ACT,  students  must  label  their  messages 
according  to  elements  of  effective  argumentation  and  requirements  for  good  collaboration.  For 
instance,  students  might  categorize  their  posts  according  to  the  type  of  message,  content  source, 
or  perceived  importance.  The  goal  of  labeling  posts  is  to  foster  problem  solving,  reflection,  and 
critical  awareness  of  one’s  ideas.  With  ACT,  students  begin  to  visualize  and  reflect  on  thinking 
patterns,  but  it  remains  an  untested  tool. 

Some  tools  more  explicitly  foster  visualization  of  online  discussion  and  interchange.  In 
ArtView,  for  example,  groups  of  learners  discuss  common  museum  images  electronically 
displayed  by  the  instructor  (Gay,  Boehner,  &  Panella,  1997).  With  this  tool,  students  can  enlarge 
images,  ask  for  additional  background  information,  engage  in  real-time  chats,  form  groups,  take 
tours  of  artwork  collections,  and  save  transcripts.  Students  here  expressed  a  preference  for 
instructors  or  guides  to  have  more  visual  authority.  In  a  study  by  Ahem  and  Repman  (1994), 
undergraduate  students  using  a  conferencing  tool  with  a  graphical  interface  (i.e.,  with  visual 
maps  of  subtopics  and  participants),  produced  more  messages  and  spent  more  time  in  the  system 
than  students  in  the  text-based  version  of  the  tool.  When  multimedia  is  added  to  such  an 
equation,  there  are  opportunities  to  archive  ideal  forms  of  performance,  replay  videoclip 
performances,  store  scanned  images,  interact  in  a  whiteboard,  hear  expert  commentary,  and  listen 
to  other  key  players  in  that  environment  (Barab  &  Duffy,  2000).  Tools  for  concept  mapping  or 
hypertextual  linking  will  play  a  vital  role  in  asynchronous  training  and  education  in  the 
upcoming  decades  (Harasim  &  Stockley,  1998). 

There  are  many  tools  that  might  be  created  to  foster  online  collaboration.  When 
collaborative  groups  use  such  tools  in  the  future,  they  will  process  information  at  a  higher  level, 
such  as  engaging  in  judgment,  seeing  multiple  perspectives,  deciding  paths  of  action,  and 
imposing  meaning  through  integrative  statements,  explanations,  and  drawing  conclusions 
(Herrington  &  Oliver,  1999).  When  sophisticated  interaction  tools  are  combined  with  authentic 
activities  student  reflection  and  articulation,  well  defined  roles,  and  timely  instructor  guidance, 
powerful  leaming  can  ensue  (Oliver,  Omari,  &  Herrington,  1998).  Of  course,  since  the  social 
interaction,  negotiation  of  knowledge,  and  resulting  learning  varies,  some  researchers  have 
advocated  for  new  and  innovative  frameworks  to  better  understand  online  learning  and 
interaction. 

Interaction  Schemes.  Many  scholars  posit  that  social  interaction  and  discourse  leads  to 
student  cognitive  development  and  higher  mental  functioning  (Vygotsky,  1978;  Wertsch,  1991). 
In  effect,  learning  is  a  social  phenomenon  wherein  students  acquire  competence  when  interacting 
with  peers  and  adults  in  a  learning  community.  From  a  sociocultural  perspective,  the  social 
activities  of  articulating,  exploring,  testing  and  refining  and  debating  ideas  can  significantly 
impact  student  thinking.  Online  conversations  between  students,  instructors,  and  expert 
professionals  or  practitioners  provide  a  forum  for  discussing  issues  and  learning  to  analyze  key 
problems  in  a  discipline  (Gay,  Boehner,  &  Panella,  1997).  Tools  and  frameworks  to  measure 
such  acts  are  detailed  in  this  section. 
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The  literature  on  distance  learning  often  breaks  interaction  into  three  types:  (1)  learner- 
content  interaction,  (2)  learner-instructor  interaction,  and  (3)  leamer-leamer  interaction.  Wagner 
(1997)  developed  a  fairly  simple  scheme  to  code  13  categories  of  interactions  found  in  distance 
learning  (see  Table  5). 


Table  5 

Interactions  Found  in  Distance  Learning 


Categories  of  Interactions  (Wagner,  1997) 

•  To  increase  learning 

•  For  negotiation  of  understanding 

•  To  increase  participation 

•  For  teambuilding 

•  To  develop  communication 

•  For  discovery 

•  To  receive  feedback 

•  For  exploration 

•  To  enhance  elaboration  and  retention 

•  For  clarification  of  understanding 

•  To  support  leamer/self-regulation 

•  To  increase  motivation 

•  For  closure 

Social  interactions.  Given  the  wealth  of  data  and  types  of  learner  interactions,  there  is  a 
need  to  look  for  important  patterns  in  the  data  (Henri,  1992;  Kuehn,  1994).  Researchers 
interested  in  social  interaction  have  explored  online  participation  patterns  and  roles,  collaborative 
knowledge  construction,  levels  of  argumentation,  group  development,  critical  thinking,  response 
complexity,  social  cues,  and  cognitive  and  metacognitive  understanding.  Henri  (1992),  for 
instance,  offers  a  popular  framework  and  analytical  model  for  better  understanding  the  learning 
process  in  computer  conferencing  environments.  She  highlights  five  dimensions  of  the  learning 
process;  namely,  participation  (e.g.,  rate,  timing,  and  duration  of  messages),  interactivity  (e.g., 
explicit  interaction,  implicit  interaction,  and  independent  comments),  social  events  (i.e., 
statements  unrelated  to  the  content),  cognitive  events  (e.g.,  clarifications,  inferences,  judgments, 
and  strategies),  and  metacognitive  events  (e.g.,  both  metacognitive  knowledge — person,  and 
task,  and  strategy  and  well  as  metacognitive  skill — evaluation,  planning,  regulation,  and  self- 
awareness).  Henri  also  offers  a  second  cognitive  model  intended  to  examine  the  depth  of 
processing,  surface  or  in-depth,  of  information  (see  Table  6).  While  this  framework  is 
comprehensive  and  informative,  some  aspects  of  this  approach  (e.g.,  measuring  metacognitive 
knowledge  in  online  discussions)  are  highly  subjective  (Hara  et  al.,  2000). 

Table  6 

Model  for  Analyzing  Online  Processing  of  Information  (adapted  and  modified  from  Henri,  1992; 
see  also  Hara  et  al.,  2000)  _ _ _ 


Surface  Processing 

In-Depth  Processing 

Repeating  information  contained  in  the 
statement  of  the  problem,  text,  or  previous 
discussion  without  making  any  inferences  or 
offering  interpretations 

Linking  facts,  ideas,  and  notions  in  order  to 
interpret,  infer,  propose,  and  judge 

Repeating  what  has  been  said  without  adding 
any  new  elements 

Offering  new  elements  of  information 

Stating  that  one  shares  the  ideas  or  opinions 
stated,  without  taking  these  further  or  adding 

Generating  new  data  from  information 
collected  by  the  use  of  hypotheses  and 
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any  personal  comments 

inferences 

Proposing  solutions  without  offenng 
explanations 

Setting  out  the  advantages  and  disadvantages 
of  a  situation  or  solution,  pros  and  cons,  etc. 

Proposing  solutions  without  a  sense  of 
implementation  criteria  and  potential  problems 

Proposing  one  or  more  solutions  with  short-, 
medium-,  and  long-term  justification 

Making  judgments  without  offering 
justification 

Making  judgments  supported  by  justification 

Asking  questions  which  invite  information  not 
relevant  to  the  problem  or  not  adding  to  the 
understanding  of  it 

Asking  questions  designed  to  provoke  content- 
related  responses  or  investigations  and  further 
discussion 

Offering  several  solutions  without  suggesting 
which  is  the  most  appropriate 

Providing  proof,  supporting  examples, 
counterexamples,  relevant  analogies  or 
metaphors 

Providing  the  situation  in  a  fragmentary  or 
short-term  manner 

Perceiving  the  problem  within  a  larger, 
connected,  or  more  long-term  perspective 

Failing  to  suggest  how  an  idea  fits  within  a 
larger  scheme  or  framework 

Developing  strategies  and  ideas  within  a  wider 
framework  or  integrative  model 

Critical  Thinking.  Closely  corresponding  to  the  cognitive  events  of  Henri’s  model, 
Garrison  (1991)  outlines  a  five-stage  model  of  critical  thinking  for  adults  resembling  a  problem 
solving  process.  These  stages  include  problem  identification,  definition,  exploration, 
applicability,  and  integration.  Garrison  argues  that  student  responsibility  and  control  of  one’s 
own  learning  are  central  to  self-directed  learning  and  critical  thinking.  He  also  attempts  to  link 
self-direction  and  collaboration  as  integral  to  the  critical  thinking  process.  In  a  way,  Garrison 
(1992)  is  hopeful  that  understanding  the  core  elements  of  critical  thinking  will  provide  a  unique 
and  important  framework  for  studying  adult  education.  Some  online  researchers  have  adapted 
Garrison’s  critical  thinking  model  to  their  analysis  of  electronic  transcripts  since  students  are 
developing  and  refining  positions,  exploring  problems,  negotiating  ideas,  questioning  the 
positions  of  others,  and  offering  problem  solutions  online  (Bakardjieva  &  Harasim,  1997).  The 
long-term  utility  of  this  scheme  is  difficult  to  predict. 

Newman,  Johnson,  Webb,  and  Cochrane  (1997)  combined  Henri’s  (1991)  and  Garrison’s 
(1991)  models  to  compare  critical  thinking  in  CMC  and  FTF  environments.  They  developed  a 
student  perception  questionnaire  to  measure  the  degree  to  which  computer  conferencing 
environments  help  arouse  student  interest  in  important  issues,  explore  and  develop  ideas, 
critically  assess  course  content  and  possible  solutions,  and  apply  course  content  to  their  own  life 
situations.  By  simplifying  the  surface  and  in-depth  ideas  of  Henri  (mentioned  above  and 
illustrated  in  Table  7;  Appendix  A)  and  also  using  Garrison’s  model,  Newman,  Johnson, 
Cochrane,  and  Webb  (1996)  created  a  coding  scheme  for  online  transcripts  addressing  statement 
relevance,  novelty,  student  knowledge  and  experience,  clarity,  idea  linking,  justification,  critical 
assessment,  practical  utility,  and  width  of  understanding  and  discussion.  While  critical  thinking 
was  evident  in  both  CMC  and  FTF  environments,  their  content  analysis  revealed  that  the  depth 
of  critical  thinking  was  higher  in  CMC  environments.  More  specifically,  CMC  students  were 
more  likely  to  bring  in  outside  information  (personal  experience,  course  materials,  etc.),  link 
ideas  and  offer  interpretations,  and  generate  important  ideas  and  solutions.  While  FTF  settings 
were  better  for  generating  new  ideas  and  creatively  exploring  problems,  CMC  fostered 
evaluation  and  interpretation  of  such  ideas. 


23 


Problem  interpretation.  Such  findings  suggest  that  whereas  FTF  might  prove  useful  in 
earlier  stages  of  problem  solving  and  critical  thinking  such  as  generating  ideas,  CMC  tends  to 
function  better  in  the  latter  stages  of  problem  interpretation  and  integration  as  well  as  idea 
linkage.  Given  these  tentative  conclusions,  researchers  might  test  whether  online  chat  tools 
could  take  the  place  of  FTF  and  equally  well  support  idea  generation,  while  asynchronous  tools 
could  be  used  for  evaluating  and  integrating  those  ideas.  In  fact,  Bonk,  Hansen,  Grabner-Hagan, 
Brown,  and  Mirabelli  (1998)  found  that  real  time  chat  tools  fostered  more  responses  per  student 
whereas  delayed  conferencing  tools  promoted  depth  and  idea  evaluation.  Not  surprisingly, 
Newman  et  al.  (1996)  recommend  that  the  next  generation  of  groupware  tools  be  designed  to 
foster  deeper  thinking  and  shared  understanding  of  topics  among  members  of  a  online  group  or 
community.  For  instance,  tools  are  needed  for  mapping  out  key  issues,  arguments,  and  positions, 
voting  and  ranking  issues,  and  forming  and  reforming  groups.  While  this  study  involved  topical 
discussions,  they  contend  that  online  group  projects  such  as  case  studies,  debates,  role  plays,  and 
symposia  (Paulsen,  1995b;  Appendix  B)  might  exhibit  Garrison’s  (1991)  five  stages  even  more 
explicitly.  Unfortunately,  Web  tools  with  such  pedagogical  components  or  features  are 
extremely  rare. 

Knowledge  construction.  In  reaction  to  frameworks  from  Henri  (1992),  Garrison  (1991), 
Levin  and  Waugh  (1998),  Newman  et  al.  (1996),  and  other  scholars  commonly  used  in  analyzing 
computer  conferencing  transcripts,  Gunawardena,  Lowe,  and  Anderson  (1997)  developed  a 
framework  based  more  on  a  learner-centered  instructional  paradigm.  Their  model  defines 
interaction  within  CMC  as  the  means  for  co-construction  of  knowledge  and  emphasizes  the 
stages  and  patterns  of  knowledge  construction  that  emerge  during  the  online  conference.  These 
researchers  are  interested  in  capturing  both  the  individual  and  social  acts  of  knowledge  creation. 
The  five  phases  in  the  social  construction  of  knowledge  in  their  model  are  detailed  in  Table  8 
(Appendix  A). 

There  are  a  number  of  codes  within  the  phases.  The  initial  study  by  Gunawardena  et  al. 
(1997)  involved  an  online  global  debate  among  hundreds  of  professional  distance  educators  prior 
to  a  conference  involving  participants  from  35  countries  across  time  zones.  Their  analyses  of  the 
debate  revealed  that  students  were  task  driven  and  competitive  in  attempting  to  win  a  debate.  As 
a  result,  the  format  of  the  debate  kept  participants  from  the  higher  levels  in  the  co-construction  of 
knowledge.  Student  dialogue  tended  to  remain  at  Phase  I  with  the  sharing  and  comparing  of 
information,  instead  of  elevating  to  knowledge  negotiation,  construction,  testing,  and  application. 
The  same  findings  were  evident  in  a  follow-up  CMC  study  of  25  managers  of  workplace  centers 
in  Canada  (Kanuka  &  Anderson,  1998).  The  authors  speculated  that  this  professional 
development  forum  was  not  an  ideal  opportunity  for  cognitive  dissonance,  discord,  and  heated 
negotiation  of  co-constructed  meaning;  but,  instead,  participants  found  greatest  value  in  sharing 
and  receiving  information  on  what  each  other  was  doing  or  found  useful.  Nevertheless,  this 
model  refocuses  instructors  and  designers  on  the  stages  in  knowledge  creation  and  the  need  to 
share  understandings  of  knowledge  among  CMC  participants. 

Online  discussion  analysis.  Somewhat  similar  to  Kanuka  and  Anderson  (1998),  Curtis 
and  Lawson  (1999)  designed  a  scheme  for  analyzing  online  discourse.  They  proposed  greater 
understanding  of  the  types  of  behaviors  typically  found  in  collaborative  learning  situations — 
such  as  giving  and  receiving  help  and  feedback,  exchanging  resources  and  information. 


explaining  and  elaborating  on  information,  sharing  knowledge  with  others,  challenging  others’ 
contributions,  advocating  increased  effort  and  perseverance  among  peers,  engaging  in  small 
group  skills,  and  monitoring  the  efforts  of  others.  Their  coding  scheme  categorizes  such  high 
level  behaviors  as  planning,  contributing,  seeking  input,  reflection  and  monitoring,  and  social 
interaction.  In  a  study  of  19  college  students  completing  three  online  assignments,  slightly  over 
a  fourth  of  their  online  behaviors  related  to  each  of  the  following  categories:  planning, 
contributing,  and  seeking  input  (see  Table  9,  Appendix  A).  Among  the  more  common  events 
were  initiating  activities,  providing  feedback,  sharing  knowledge,  seeking  feedback,  and 
reflecting  on  the  medium.  As  in  other  studies  (see  Bonk,  Malikowski,  Angeli,  &  Supplee,  1998), 
few  students  challenged  others  or  attempted  to  explain  or  elaborate  on  their  particular  positions. 
Curtis  and  Lawson  (1999)  also  confirmed  common  findings  of  student  resistance  to  peer  critique 
as  well  as  the  limited  off  task  behaviors  of  around  5  percent  in  such  activities  (Bonk,  Hansen,  et 
al.,  1998;  Cooney,  1998).  They  recommend  that  instructors  promote  greater  online  debate 
perhaps  by  modeling  appropriate  ways  to  challenge  others. 

Taking  a  more  mathematical  approach,  Hara  (2000)  recommends  Formal  Concept 
Analysis  (FCA)  for  understanding  conceptual  hierarchies  in  e-leaming.  FCA  is  based  on  a 
mathematical  lattice  theory  that  analyzes  quantitative  data  visually.  According  to  Hara  (2000),  it 
can  be  used  to  describe  social  relationships.  For  instance,  she  used  it  to  reveal  complex 
relationships  among  categories  of  coded  data  in  online  environments,  thereby  providing  insights 
into  online  interactions. 

A  simpler  scheme  was  used  by  Hoffman  and  Elliot  (1998)  who  coded  Web  dialogue 
according  to  the  six  levels  of  Bloom’s  taxonomy.  Interestingly,  these  researchers  also  coded 
affective  indicators  such  as  supporting  comments,  affective  sharing,  values,  praise,  empathy,  and 
constructive  judgments.  They  found  that  student  Web  electronic  dialogue  occurred  at  a  deeper 
level  than  their  more  superficial  written  journals.  These  researchers  concluded  that  case-based 
discussions  on  the  Web  can  foster  student  problem  solving,  interaction,  and  the  creation  of  a 
network  of  peers  with  whom  to  communicate. 

Interpersonal  considerations.  In  contrast  to  critical  thinking  and  metacognition, 
researchers  such  as  Walther  (1992,  1996)  are  interested  in  interpersonal  aspects  of  CMC  and 
how  interaction  changes  over  time.  Walther  argues  that  there  is  less  social  information  per 
message  in  CMC  due  to  limited  nonverbal  cues.  When  there  is  minimal  time,  limited  turn 
taking,  and  anonymous  posting,  CMC  groups  tend  to  be  more  task  focused,  democratic,  and 
impersonal  (Walther,  1996).  Over  time,  users  can  share  ideas  and  manage  relationships,  thereby 
building  impressions  and  interpersonal  relationships  with  each  other.  Given  sufficient  time  and 
message  exchange,  CMC  participants  can  develop  similar  communication  relationships  to  FTF 
environments  (Walther,  1992).  In  fact,  as  group  intersubjectivity  grows,  he  posits  that  CMC 
environments  can  develop  “hyperpersonal”  relationships  that  might  not  occur  in  FTF  settings. 
Individuals  engaged  in  multi-tasking  when  using  chat  tools,  for  instance,  experience  greater 
intensity  of  interaction  than  is  possible  in  FTF  situations,  even  though  most  CMC  technologies 
are  not  prime  vehicles  for  coherent  conversations  (Herring,  1 999;  Appendix  B).  He  even  finds 
that  the  time  stamping  of  a  message  (night  or  morning)  conveys  significant  nonverbal  cues, 
thereby  affecting  one’s  perception  of  sender  dominance  (e.g.,  talking  business  during  off-hours) 
and  intimacy  (e.g.,  slow  replies  to  a  social  message)  (Walther  &  Tidwell,  1995).  Given  the 
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accumulation  of  vast  message  exchanges  over  time  (Slatin,  1 993),  increasing  interpersonal 
effects  are  often  found  in  computer  conferencing  environments  (Kang,  1998). 

Bakardjieva  and  Harasim  (1997)  also  argue  that  well  designed  educational  computer 
conferences  represent  an  important  example  where  such  socially  enhanced  learning  can  occur. 
Given  the  impact  of  social  interaction  on  cognitive  thought,  there  is  a  need  to  understand  the 
forms  of  talk— social,  cognitive,  off-task,  etc.— that  might  evolve  during  online  interactions. 
These  researchers  examined  both  the  cognitively  charged  speech  acts  (e.g.,  identifying  problems, 
stating  positions,  questioning  positions,  etc.)  as  well  as  interactively  charged  speech  acts  (e.g., 
acknowledgments,  support,  disagreements,  etc.)  of  online  discourse  in  college  courses  (see  Table 
10).  Three  of  the  most  common  cognitive  acts  they  discovered  were  identifying  problems, 
arguing  positions,  and  offering  solutions  or  conclusions.  While  interactivity  was  more  varied, 
they  found  a  high  level  of  mutual  support,  including  acknowledgments,  encouragement,  personal 
information  and  feelings,  and  meta-interaction. 


Cognitive  and  interactive^  acts  m  online  discourse  (Bakardjieva  &  Harasim ,  1997) 


Subgroups 

Cognitive  Acts 

Subgroups 

Interactive  Moves 

Interrogation  Zone 

1 .  Identifying  Problem 

2.  Exemplifying  Problem 

3 .  Introducing  Rel  ated 
Problems 

4.  Linking  Problems 

Thinkers’  Relations 

1 .  Support 

2.  Encouragement 

3.  Acknowledgement 

4.  Building  on 

5.  Negotiation 

6.  Partial  Agreement 

Analysis  Zone 

5.  Analyzing  Problems 

6.  New  Perspective  to 
Problem 

7.  Defining  Problem 

7.  Disagreement 

8.  Challenge 

Statement  Zone 

8.  Providing  Information 

9.  Arguing  Position 

10.  Providing  Evidence  to 
Justify  Position 

Personal  Relations 

9.  Personal 
Information/Reflection 

10.  Revealing  Personal 
Feelings 

1 1 .  Personal  Address 

Critique  Zone 

1 1 .  Comparing  Positions 

12.  Questioning  Position 

13.  Opposing  Position 

Group  Relations 

12.  Coordination 

13.  Meta-interaction 

14.  Phatic  Communication 

Metacognitive  Zone 

14.  Metacognitive  Act 

Closure  Zone 

15.  Drawing  Conclusion 

1 6.  Offering  Solutions 

17.  Challenging 
Conclusion/Solution 

Bakardjieva  and  Harasim  concluded  that  these  online  conferences  blended  both  cognitive 
and  interactive  acts.  In  effect,  students’  stated  positions  while  supporting  other  points  of  view, 
questioned  positions  while  inviting  others  to  comment,  and  drew  personal  conclusions  while 
building  on  previous  comments.  Nevertheless,  while  there  were  high  levels  of  mutual  support  in 
all  groups  and  some  degree  of  personalization,  these  same  students  tended  to  avoid  controversy 
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and  critical  attitudes  toward  their  peers.  The  researchers  argued  that  greater  intersubjectivity  is 
needed  in  online  conferences  wherein  participants  agree,  disagree,  challenge,  and  negotiate  with 
their  peers  with  whom  they  share  personal  knowledge,  interests  or  history. 

Sociocultural  considerations.  Bonk  and  his  colleagues  have  analyzed  computer 
conferencing  and  e-leaming  from  a  sociocultural  perspective  (Bonk  &  King,  1998).  The  goal  of 
this  research  is  to  foster  student-student  and  student-instructor  construction  and  negotiation  of 
meaning  by  extending  the  class  beyond  typical  time,  geography,  and  space  limitations.  In  a 
series  of  studies,  Bonk  and  his  colleagues  have  looked  at  the  activity  setting  of  these  new 
environments  (Bonk  &  Sugar,  1998;  Kirkley  et  al.,  1998).  Variables  of  interest  have  included 
the  types  of  talk,  forms  of  mentoring,  levels  of  questioning,  forms  of  participation  (Zhu,  1998), 
patterns  of  interaction,  and  levels  of  scaffolding  in  various  online  environments.  Their  research 
has  revealed  that  direct  instruction  is  just  one  of  at  least  twelve  forms  of  online  assistance  or 
mentoring  (Table  11,  Appendix  A)  provides  a  scheme  for  considering  the  forms  of  online 
learning  assistance).  Across  these  studies,  one  senses  a  a  shift  away  from  modeling  and  direct 
instruction  to  task  structuring,  questioning,  scaffolding,  and  feedback  (Bonk  &  King,  1998; 

Bonk,  Malikowski,  Angeli,  &  East,  1998;  Bonk,  Malikowski,  Angeli,  &  Supplee,  1998).  The 
instructor  role  varies  tremendously  with  direct  instruction  playing  a  far  smaller  part  than  in 
conventional  classroom  instruction. 

These  researchers  have  also  found  that  pedagogical  activities  on  the  Web  can  foster  high 
as  well  as  low  levels  of  student  questioning  (Bonk  &  Sugar,  1998),  limited  off-task  behaviors 
(Bonk,  Hansen  et  al.,  1998;  Cooney,  1998),  and  increasing  group  cohesive  and  student-centered 
environments  over  time  (Hara  et  al.,  2000;  Kang,  1998).  And  while  participation  patterns  change 
from  the  regular  classroom,  discussion  is  not  entirely  equal  in  electronic  forums.  Nonetheless,  in 
Cooney’s  (1998)  study  of  a  high  school  English  classroom,  the  quantity  and  quality  of  discussion 
was  elevated  significantly  for  all  students  in  an  electronic  network.  At  the  same  time,  the 
limited  off  task  behavior  found  in  Cooney’s  study  and  in  most  of  this  research  may  not  be  a 
positive  finding.  In  fact,  the  creation  of  online  communities  is  difficult  since  e-leaming  students 
are  extremely  task  driven.  An  additional  problem  is  that  student  dominance  and  antagonism  is 
rare.  When  it  does  occur  or  when  roles  like  pessimist  and  devil’s  advocate  are  assigned,  there 
tends  to  be  more  social  interaction  and  dialogue  (Bonk,  Hansen  et  al.,  1998).  Not  surprisingly, 
controversial  and  “hot”  topics  do  provoke  the  most  discussion  (Bonk  et  al.,  2000).  And  while 
student  case  discussions  online  generally  lack  justification  (Bonk,  Malikowski,  Angeli,  & 
Supplee,  1998),  adding  students  from  other  countries  or  universities  to  the  discussions  increases 
the  chances  for  students  to  link  arguments  to  textbook  concepts  and  terminology.  Finally,  CMC 
environments  can  support  the  social  construction  of  knowledge  (Zhu,  1998). 

Conferencing  dialogues.  In  contrast  to  sociocultural  theory,  Howell-Richardson  and 
Mellar  (1996)  analyzed  an  online  course  for  educational  trainers  using  Speech  Act  Theory  to 
analyze  the  conferencing  dialogue.  Here,  they  looked  at  the  structural  or  illocutionary  properties 
(e.g.,  interrogative,  declarative,  directive,  elicitation),  group  focus,  task  focus,  addressee,  and 
intermessage  referencing.  They  found  that  minor  changes  in  the  online  moderating  facilitated 
vastly  different  patterns  of  interaction.  Not  surprisingly,  moderators  taking  on  a  role  of 
facilitator  with  short  organizational  and  group  cohesion  messages  tended  to  generate  more 
intermessage  referencing  than  moderators  who  acted  as  authorities. 
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These  findings  are  consistent  with  studies  that  show  that  CMC  also  fosters  informal  and 
exploratory  conversation  (Weedman,  1999)  that  allows  students  and  instructors  to  take  risks  and 
share  knowledge.  In  a  study  of  80  college  undergraduates,  Ahem,  Peck,  and  Laycock  (1992) 
also  found  that  a  conversational  style  of  interaction  from  the  instructor  produced  higher  and  more 
complex  levels  of  student  participation.  When  online  instructors  were  more  informal  and 
spontaneous  in  their  commenting  here,  students  were  more  interactive  with  each  other,  compared 
to  conditions  wherein  the  instructor  simply  posed  formal  topic-centered  statements  or  questions. 
In  effect,  responding  to  teacher  questions  or  statements  online  is  simply  an  extension  of  the 
recitation  method;  the  more  teacher-centered  the  e-leaming  environment,  the  less  student 
exploration,  engagement,  and  interaction.  As  Tharp  and  Gallimore  (1988)  demonstrated  with 
their  highly  acclaimed  “instructional  conversation”  method,  students  need  to  be  invited  into  the 
discourse  through  many  ways  of  instructor  and  peer  assistance. 

Understanding  online  interaction  seems  key.  Moore  and  Kearsley  (1996)  observe  the 
nature  and  extent  of  interaction  will  vary  according  to  the  organizational  needs,  designer  goals 
and  teaching  philosophies,  nature  of  the  subject  matter,  level  of  student,  media  selected,  and 
student  location  and  culture.  Garton,  Haythomthwaite,  and  Wellman  (1997)  argued  that 
examining  relations  within  network  groups  allows  researchers  to  track  how  groups  form  and 
function.  They  recommend  shifting  the  focus  from  individual  or  small  group  performance  with 
technology  to  begin  studying  social  networks.  In  effect,  the  unit  of  analysis  for  such  scholars  is 
the  relationship  among  people.  Consequently,  they  evaluate  the  content,  direction,  and  strength 
of  relations  as  well  as  the  ties  that  connect  one  or  more  relations. 

While  social  interaction  is  often  declared  vital  to  e-leaming  success,  scholars  have 
difficulty  agreeing  on  what  is  interaction  (Wagner,  1994).  Rafaeli  and  Sudweeks  (1997),  for 
instance,  consider  interactivity  as  the  extent  to  which  messages  in  a  sequence  relate  to  each 
other.  Their  analyses  of  network  conferencing  in  platforms  available  to  the  public  (Bitnet, 

Usenet,  and  CompuServe)  found  that  slightly  over  50  percent  of  these  messages  were  generally 
reactive  statements,  referring  to  just  a  single  message  in  front  of  them.  Only  around  10  percent 
were  truly  interactive.  Most  messages  contained  factual  statements  or  opinions  while  many  also 
contained  questions  or  requests.  In  contrast  to  hypotheses  that  frequent  participators  would  be 
more  interactive,  these  group  members  were  more  reactive  than  low  participators.  When 
messages  were  interactive,  they  were  more  opinionated  and  humorous,  while  containing  more 
self-disclosure  as  well  as  a  sense  of  involvement  and  belonging.  Not  surprisingly,  people  are 
attracted  to  fun,  open,  frank,  helpful,  and  supportive  environments. 

Given  the  above  findings,  it  is  critical  to  understand  the  degree  to  which  different  types  of 
tools  and  tasks  foster  the  social  negotiation  of  meaning.  Just  how  interactive  and  collaborative 
are  online  environments?  It  also  might  be  interesting  to  understand  how  different  interactions 
impact  different  types  of  students  and  learning. 

Learning  Styles 

Not  surprisingly,  scholars  are  curious  whether  different  ways  of  perceiving  and 
processing  information  can  impact  student  success  when  learning  on  the  Web.  Given  the  fact 
that  not  all  students  leam  the  same — some  prefer  stories,  some  observation,  some  hands  on 


experience,  etc. — there  is  growing  interest  in  how  the  Web  facilitates  learners  from  distinct 
learning  styles  and  forms  of  intelligence.  Nelson  (1998)  argues  that  different  Web  tools  facilitate 
different  types  of  intelligence.  For  instance  chat  tools  might  address  interpersonal  and  linguistic 
intelligence.  In  contrast,  computer  conferencing  tools  might  foster  linguistic,  spatial,  and 
interpersonal  intelligence,  whereas  e-mail  might  play  a  similar  role  but  also  influence 
intrapersonal  intelligence.  Finally,  class  Web  sites  might  address  nearly  all  forms  of  intelligence 
by  incorporating  video  and  sound  clips,  games  and  simulations,  class  notes,  graphics,  tutorials, 
etc.  Unfortunately,  as  noted  twice  before,  most  class  Web  sites  utilize  fairly  basic  technologies. 

In  terms  of  the  impact  of  e-leaming  on  different  learning  styles,  one  must  keep  in  mind 
that  extensive  hypermedia  research  from  the  early  1990s  had  mixed  results.  Dillon  and 
Gabbard’s  (1998)  review  of  the  literature  included  one  study  where  field  independent  learners 
outperformed  field  dependent  learners  on  all  tasks.  Other  studies  showed  differences  in 
cognitive  styles  and  learning  time,  locations  visited,  and  tasks  completed.  However,  Shih  et  al., 
(1998)  found  no  differences  in  learning  styles  on  the  Group  Embedded  Figures  Test  and  Web- 
based  learning  achievement.  And  while  Reed  and  Oughton  (1997)  determined  that  field 
dependent  learners  tend  to  take  more  linear  learning  paths  in  comparison  to  the  nonlinear 
navigation  of  field  independent  learners,  such  differences  dissipated  over  time.  In  Kerka’s 
(1998)  brief  review  of  the  research,  she  indicated  that  field  independent  learners  tend  to  perform 
more  efficient  searches,  take  less  time,  and  are  more  comfortable  in  open-ended  Web 
environments,  whereas  field  dependent  learners  may  require  site  maps  and  global  overviews. 
Since  the  Web  is  a  hypermedia  environment,  all  these  studies  have  relevance  here. 

It  is  important  to  know  where  different  types  of  learners  spend  their  time  in  Web 
environments  and  succeed  in  learning  (Kim,  1 999).  How  often  do  different  types  of  learners  use 
a  particular  tool  or  Web  site  (i.e.,  what  are  their  navigation  paths)?  How  much  time  is  spent  at 
each  node  or  completing  each  task?  And  how  much  structure  might  improve  learning  for 
different  types  of  learners?  These  and  other  questions  leave  many  gaps  in  understanding 
e-leaming. 


Online  Communities 

As  a  large  and  complex  organization,  the  Army  has  many  communities,  from 
installations,  branches,  and  career  fields  to  year  groups,  special  interest  groups,  and  veterans 
clubs.  An  informal  examination  of  military  chat  rooms  on  the  Internet  revealed  over  53  active 
rooms  (Olson,  personal  communication.)  A  challenge  is  to  channel  these  natural  interests  in 
organizational  communities  into  learning  communities  that  can  enhance  the  development  of 
skills  and  acquisition  of  knowledge  by  soldiers.  This  section  examines  the  literature  on  the 
formation  and  membership  of  learning  communities  in  educational  settings,  provide  examples, 
and  considers  the  online  class  as  a  learning  community. 

There  is  a  huge  void  in  the  research  regarding  how  online  learning  communities  are 
formed.  In  a  1994  Presidential  address  to  the  American  Educational  Research  Association, 
Brown  (1994)  outlined  key  learning  principles  of  the  cognitive  revolution  of  the  past  thirty  years. 
She  also  pointed  to  the  need  for  active  and  reflective  learning  in  a  community  of  discourse  and 
community  of  practice.  Brown  argued  that  “Learning  and  teaching  depend  on  creating. 
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sustaining,  and  expanding  a  community  of  research  practice.  Members  of  the  community  are 
critically  dependent  on  each  other.  No  one  is  an  island;  no  one  knows  it  all;  collaborative 
learning  is  not  just  nice,  it  is  necessary  for  survival”  (Brown,  1994,  p.  10).  It  is  time  to  explore 
how  new  technology  environments  can  facilitate  the  formation  of  such  communities. 

Not  only  does  learning  theory  indicate  a  need  for  online  communities,  but  there  are  many 
practical  reasons  as  well.  For  example,  the  President  of  the  University  of  North  Carolina,  Broad 
(1999)  points  out  that  the  formation  of  communities  is  one  of  the  key  indicators  of  high  quality 
education  in  residential  college  settings.  Since  the  most  powerful  learning  in  university  settings 
is  achieved  in  faculty-student  and  student-student  interactions  and  mentoring  situations.  Broad 
asks  how  those  learning  community  features  can  be  replicated,  extended,  and  transformed  from  a 
distance.  She  also  asks  what  levels  of  interaction  are  desirable  and  essential  and  how  the  levels 
of  interaction  might  differ  by  domain  and  institutional  level? 

Despite  such  prodding,  minimal  research  exists  on  the  technological  and  pedagogical 
variables  needed  to  foster  virtual  communities.  Research  proposed  later  in  this  paper  is  intended 
to  address  this  gap  in  the  literature.  Unfortunately,  even  as  institutions  and  instructors  hurriedly 
place  courses  on  the  Web,  the  tools  for  e-leaming  interaction  are  not  fully  developed  and  thought 
out  (Hughes  &  Hewson,  1998). 

Designers  of  online  training  and  education  need  to  understand  the  factors  underlying 
successful  online  learning  communities.  A  number  of  scholars  have  begun  to  describe  the 
factors.  For  example,  Bielaczyc  and  Collins  (1998)  argue  that  learning  cultures  need 
membership  with  diverse  expertise,  mechanisms  for  sharing  one’s  learning,  common  goals  that 
help  continually  advance  its  collective  knowledge  and  skills,  and  a  value  on  learning  how  to 
leam.  Each  member  may  have  different  knowledge  or  identities  that  the  online  community 
should  advertise  and  utilize.  There  also  is  a  common  language  or  socially  shared  knowledge 
base  for  describing  and  promoting  ideas,  processes,  plans,  and  goals.  Through  shared  discourse, 
members  formulate  and  exchange  ideas.  The  tools  developed,  therefore,  should  help  in 
advancing  learning. 

Unlike  most  research  on  learning,  there  is  a  dual  focus  here  on  both  individual  knowledge 
growth  and  joint  products  or  collective  efforts.  From  such  a  perspective,  it  is  important  to 
identify  the  factors  that  foster  or  negate  community  building  in  e-leaming  environments.  How 
are  learning  communities  designed  and  supported?  What  technology  tools  support  distributed 
interactive  learning  communities?  How  can  distance  learning  technology  be  used  to  build  online 
communities  (Hiltz,  1998)?  Should  similar  design  principles  and  tools  be  used  in  both  training 
and  education  environments?  How  might  they  differ?  Answer  to  these  questions  will  not  come 
without  significant  research  inquiry  and  support. 

Community  Membership 

Before  online  communities  can  be  understood  and  supported,  it  is  vital  to  know  what 
qualifies  as  a  community.  Community  psychologists  suggest  that  there  are  four  key  factors  for  a 
sense  of  community:  (1)  membership,  (2)  influence,  (3)  fulfillment  of  individual  needs,  and  (4) 
shared  events  and  emotional  connections  (McMillan  &  Chavis,  1986).  In  terms  of  membership, 
a  sense  of  belonging,  community  boundaries,  identity,  and  personal  investment  all  contribute  to 


membership  in  a  community.  Membership  also  tends  to  create  a  sense  of  cognitive  dissonance 
associated  with  one’s  responsibility  to  sacrifice  for  the  community,  thereby  enhancing  member 
confidence,  sense  of  entitlement,  and  loyalty  to  the  group  (McMillan,  1996).  The  second  key 
factor,  influence,  may  include  influencing  the  community  as  well  as  being  influenced  by  it.  The 
notion  of  influence  also  instills  some  pressure  for  uniformity  and  conformity  that  spurs  even 
greater  member  closeness.  Third,  communities  provide  rewards  and  reinforcers  that  fulfill 
personal  needs  and  are  critical  to  staying  within  the  community.  Fourth,  members  have 
emotional  bonds  from  shared  histories  that  connect  members  and  encourage  continued 
investment  and  involvement  in  the  community.  McMillan  and  Chavis,  in  fact,  developed  a  sense 
of  community  model  that  summarizes  how  the  subelements  work  together  to  create  and  maintain 
communities.  To  test  theories  and  models,  there  are  “Sense  of  Community”  scales  and  indices  to 
determine  the  relative  influence  of  each  factor  (Chavis,  Hogge,  &  McMillan,  1986). 

Based  on  the  above  model  from  McMillan  and  Chavis  (1986),  Chao  (1999)  recently 
designed  a  categorization  scheme  for  online  communities.  Online  indicators  of  factor  one — 
membership — include  self-disclosure  statements,  acknowledging  other’s  membership,  the  paying 
of  dues  in  terms  of  time  and  energy,  references  to  the  boundaries  of  the  community,  and 
attempting  membership.  Online  influence  might  occur  when  referring  to  norms,  rules,  laws,  or 
other  orders,  attempting  to  influence  or  persuade  others,  being  influenced  by  others,  and 
identifying  and  trusting  some  authority.  Fulfillment  of  individual  needs  is  found  when  one  is 
attempting  to  find  common  ground,  express  a  personal  need,  thank  or  acknowledge  someone  for 
needed  information,  give  and  receive  information,  or  voice  criticisms,  suggestions,  or  differences 
of  opinion.  Finally,  sharing  events  and  emotional  experiences  as  well  as  identifying  the  spiritual 
bond  of  the  group  might  occur  when  referring  to  stories  of  what  has  happened  in  the  past  and 
using  special  symbols  or  language  specific  to  members  of  the  group.  While  still  in  the 
development  phase,  such  a  categorization  scheme  might  eventually  help  identify  events  that 
contributed  to  the  development  of  an  online  community  in  both  training  and  education  settings. 

It  is  important  to  not  only  advance  the  types  of  tools  for  building  virtual  learning 
environments,  but  there  also  is  a  pressing  need  to  investigate  how  online  communities  are 
formed  and  sustained.  What  does  an  online  learning  community  do?  What  are  the  key 
principles  that  help  create  a  sense  of  learning  community  (Barab  &  Duffy,  2000;  McMillan, 
1996;  McMillan  &  Chavis,  1986;  Schwier,  1999)?  Schwier  argues  that  electronic  learning 
environments  too  often  fail  to  develop  a  community  of  learners.  He  then  provides  the  following 
seven  guidelines  for  the  development  of  virtual  learning  communities: 

1 .  Historicity  Element:  Incorporate  prior  experiences  and  stories  of  members,  explicitly 
share  the  culture  and  values  of  the  virtual  community,  and  make  public  the  history  of  the 
community. 

2.  Identity  Element:  Foster  team-building  activities,  develop  community  logos,  publicly 
acknowledge  accomplishments  of  groups  and  members,  and  articulate  the  community  focus, 
purpose,  and  requirements  for  membership. 

3.  Mutuality  Element:  Include  group  exercises,  assignments,  and  activities  that  require  each 
member  to  contribute  to  the  final  product,  while  also  inserting  questions  and  guidance  that 
encourage  members  to  share  solutions  and  invest  in  each  other’s  ideas  and  concerns. 
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4.  Plurality  Element:  Encourage  membership  in  and  participation  from  groups  and 
professional  associations  related  to  the  learning  focus,  including  those  from  other  countries 
or  locales  pursuing  similar  goals  and  issues. 

5.  Autonomy  Element:  Within  group  identity,  it  is  vital  to  foster  individual  expression, 
promote  respectful  communication  of  ideas,  and  create  strategies  for  settling  disputes  and 
reaching  consensus. 

6.  Participation  Element:  Allow  group  members  the  chance  to  shape  learning  agendas,  while 
guiding  the  participation  of  new  members,  promoting  individual  or  group  exploration  outside 
of  the  key  learning  focus,  and  encouraging  risk  and  outside  lurking. 

7.  Integration  Element:  Articulate  a  common  set  of  beliefs  or  group  norms  as  they  emerge 
within  a  learner-centered  philosophy  and  pedagogy  that  celebrates  individual 
accomplishments  while  building  group  identity  and  momentum. 

Communities,  Schwier  (1999)  argues,  are  collections  of  people  bound  together  by  some 
common  reason.  A  learning  community,  therefore,  is  a  group  of  individuals  who  are  interested 
in  a  common  topic  or  area  and  who  engage  in  knowledge-related  transactions  as  well  as 
transformations  within  it  (Fulton  &  Riel,  1999).  A  community  of  learners  is  apparent  when 
learners  know  and  value  each  other,  discuss  common  interests,  support  each  other’s  needs,  share 
control  and  responsibility,  and  take  risks  in  a  trusting  atmosphere  (McLoughlin  &  Oliver,  1999). 
They  take  advantage  of  the  opportunity  to  exchange  ideas  and  learn  collectively.  He  further 
contends  that  certain  conditions  need  to  exist  for  nurturing  a  learning  community  such  as  a  leader 
setting  the  tone,  transparent  technologies  to  foster  task  completion  and  the  development  of 
interpersonal  relationships,  a  safe  and  comfortable  environment  for  participation,  and  an 
emphasis  on  narratives  and  story  telling.  He  then  describes  how  these  elements  play  a  role  in  the 
formation,  maturation,  and  decline  or  metamorphosis  of  the  learning  community. 

Kulp  (1999),  in  a  white  paper  related  to  LeamingSpace  from  IBM,  suggests  the  need  for 
goals  and  milestones  for  the  group  to  work  toward.  In  effect,  those  in  the  learning  community 
need  to  build,  problem  solve,  invent,  create,  and  co-leam.  Both  experienced  learners  and  novices 
should  support  each  other  through  interaction  and  negotiation  of  ideas.  Novices  might  take  on 
more  of  an  observer  role  initially  as  they  are  apprenticed  into  the  community  of  practice.  A 
novice  gradually  appropriates  the  skills  necessary  for  him  or  her  to  become  a  competent  and 
skillful  member  of  the  community.  He  or  she  becomes  acclimated  to  the  setting  by  observing  as 
a  legitimate  peripheral  participant  within  the  community  and  gradually  taking  on  a  more  critical 
role  (Lave  &  Wenger,  1991).  Such  newcomers  eventually  become  old-timers.  In  essence,  a 
community  of  practice  is  a  place  wherein  resources  and  specific  expertise  are  shared  and  made 
accessible  to  others. 

The  learning  community  must  bring  people  together  for  some  initial  common  interest  or 
quest;  e.g.,  sharing,  problem  solving,  collaborating,  or  learning.  Similarly,  there  is  a  need  for  a 
common  reference  point  for  the  online  group  as  well  as  multiple  entrance  points  (Duffy, 
McMullen,  Barab,  &  Keating,  1998).  The  cultural  and  historical  heritage  of  the  community  of 
practice  will  normally  include  shared  goals,  opportunities  for  negotiating  ideas,  and  common 
practices  or  rituals  (Barab  &  Duffy,  2000).  When  those  components  are  in  place,  there  is  greater 
opportunity  for  individuals  to  function  within  an  interdependent  system  and  for  new  members  to 
work  beside  and  leant  from  more  competent  members.  New  members  inherit  the  goals, 
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practices,  and  rituals  of  the  previous  members,  and,  over  time,  such  newcomers  will  replace  the 
old  timers.  So,  there  are  some  common  goals  and  values,  feelings  of  commitment  and  trust,  and 
something  valuable  that  binds  the  participants.  In  addition,  members  must  have  opportunities  to 
contribute  to  and  develop  the  online  community.  As  such,  members  of  the  community  have 
influence  on  the  direction  of  the  community  and  new  membership. 

Online  Learning  Community  Examples 

So,  how  does  one  create  an  online  learning  community?  Barab  and  Duffy  (2000)  indicate 
an  initial  need  for  a  mission  statement,  purpose,  or  common  reference  point.  Second,  there  must 
be  meaningful  membership  wherein  one’s  questions  and  needs  are  addressed  and  members  can 
learn  about  each  other.  At  the  same  time,  there  usually  is  some  type  of  learning  facilitator  or 
conversational  guide  that  focuses  and  refocuses  the  group.  In  addition,  online  communities 
benefit  from  separate  spaces  or  rooms  for  sharing  information  and  for  socializing  and  creating 
interesting  spaces.  For  example,  there  might  be  regularly  scheduled  types  of  events  in  those 
rooms  or  meeting  halls.  One  might  also  change  the  Web  site  to  connote  seasonal  changes  during 
the  year  or  provide  other  means  for  the  feeling  of  time  passages.  New  or  prospective  members 
also  might  be  guided  within  the  site  with  chats,  tours,  and  visitor  guidelines  that  welcome  them 
to  the  online  community.  Members  might  even  hold  positions  or  responsibilities  within  the 
different  areas  of  the  community. 

TAPPED  IN  is  an  environment  for  teacher  professional  development  and  informal 
collaborative  activities  (Schlager  &  Schank,  1997).  This  resource  combines  opportunities  for 
informal  and  formal  learning  that  emphasized  collaboration  and  social  interaction  within  a 
supportive  community  of  practice.  In  the  late  1990s,  there  were  over  6,000  K-16  teachers,  staff, 
and  researchers  within  the  TAPPED  IN  environment.  Here,  teachers  with  diverse  skills  and 
interests  can  meet  at  any  time,  leam  about  many  educational  reform  ideas  and  approaches,  and 
find  useful  materials  and  resources  (Appendix  B).  Members  hold  real-time  discussions  and 
classes,  browse  Websites  collectively,  explore  professional  development  options,  and  interact  via 
mailing  lists  and  discussion  boards  all  in  a  single  venue.  Instead  of  heavily  relying  on  video 
conferencing  or  asynchronous  discussions,  TAPPED  IN  is  primarily  a  synchronous  environment 
relying  on  a  multi-user  virtual  environment.  Filled  with  different  floors,  offices,  and  meeting 
rooms,  members  can  name  and  furnish  these  rooms,  create  and  share  documents  and  hyperlinked 
objects,  and  post  items  in  their  own  workrooms.  By  employing  Java  technology,  TAPPED  IN 
rooms  are  dynamically  and  continuously  updated  by  the  participants  of  the  community.  In 
effect,  TAPPED  IN  helps  overcome  teacher  isolation  by  providing  a  rich  sharing  of  experiences 
and  resources  while  also  recognizing  and  rewarding  participant  achievements.  In  addition,  it  is  a 
generally  safe  and  supportive  environment  for  learning. 

Online  Classes  as  Learning  Communities 

In  contrast  to  the  permanent  environments  such  as  TAPPED  IN,  Barab,  Thomas,  and 
Merrill  (1999)  analyzed  how  a  sense  of  community  might  be  formed  in  an  online  graduate  course 
in  adult  education.  The  instructor  of  this  particular  course  was  explicitly  intending  to  create  an 
online  learning  community,  instead  of  an  environment  wherein  the  instructor  dictated  the 
content.  Consequently,  the  course  modules  included  opportunities  for  collaborative  explorations 
and  shared  personal  experiences.  Barab  et  al.  (1999)  found  that  online  courses  can  support  the 
development  of  a  learning  community  when  they  (1)  can  flexibly  accommodate  diverse  learner 
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needs  and  interests;  (2)  foster  the  co-construction  of  meaning  through  information  sharing;  (3) 
allow  for  student  stories  which  are  personal  or  filled  with  self-disclosures;  and  (4)  create  a 
positive,  warm,  and  psychological  safe  environment  for  learning.  They  also  point  out  that, 
according  to  the  adult  education  literature,  it  is  important  for  learners  to  apply  course  content  to 
their  lived  experiences  and  personal  situations.  This  was  crucial  to  the  emergence  of  a  learning 
community  here  since  student  identity  and  personal  development  could  co-evolve  with  course 
participation  and  increasing  competence  with  course  material.  Qualitative  analyses  of  student 
posts  and  later  member  checking  indicated  that  the  design  of  an  open,  flexible,  and  inviting 
climate  for  learning  was  central  to  the  evolution  of  this  community.  The  authors  caution  that 
courses  involving  less  personal  material  or  more  technical  content  may  not  foster  as  much 
personal  interaction  and  sharing  of  experiences. 

What  about  more  technical  courses?  Phelps,  Ashworth,  and  Hahn  (1991)  discovered  that 
asynchronous  computer  conferencing  in  military  settings  can  increase  student  camaraderie, 
cohesion,  connectedness,  and  sense  of  accomplishment.  Such  consistency  in  training  and 
education  environments  is  an  important  finding. 

In  attempting  to  foster  a  professional  development  community,  Kanuka  and  Anderson 
(1998)  developed  a  survey  instrument  wherein  participants  were  asked  to  indicate  their 
agreement  with  statements  related  to  the  construction  of  knowledge  and  the  creation  of  online 
learning  communities.  Similarly,  Bonk,  Oyer,  and  Medury  (1995)  created  an  instrument  to 
explore  the  degree  of  social  constructivism  students  and  instructors  perceive  and  prefer.  Table 
12  (Appendix  A)  combines  items  from  these  two  scales  to  create  an  entirely  new  scale,  the 
“ Social  Constructivism  and  Learning  Communities  Online"  (SCALCO)  scale  for  measuring 
student  online  learning.  Understanding  student  perceptions  of  the  constructivist  nature  of  the  e- 
leaming  environment  may  prove  informative  to  both  researchers  and  instructors.  Just  what 
aspects  of  a  Web  course  are  critical  for  creating  a  learner-centered  environment?  Is  there  a  gap 
between  instructor  and  student  perception  of  the  learning  environment  and  between  reality  and 
ideal  world?  Will  training  or  education  e-leaming  environments  be  deemed  more 
constructivistic  in  nature  than  traditional  classroom  settings? 

In  a  more  recent  study,  Rourke,  Anderson,  Garrison,  and  Archer  (2000)  pointed  to  ways 
for  fostering  knowledge  construction  through  critical,  reflective  discourse.  They  explored  the 
social  presence  or  ability  for  participants  to  project  themselves  socially  and  emotionally  into  the 
online  community.  Factors  coded  for  social  presence  here  included  reinforcing  behaviors  (e.g., 
complimenting,  expressing  agreement),  interactive  behaviors  (e.g.,  continuing  a  thread,  quoting 
from  other  messages,  explicitly  referring  to  other  messages  or  others  by  name,  asking  questions, 
and  referring  to  the  group  as  a  whole),  and  affective  behaviors  (expressing  emotions,  humor, 
salutations,  greetings,  and  self-disclosures).  Rourke  et  al.  found  approximately  20  interactive 
behaviors  and  12  affective  behaviors  for  every  1,000  words  of  text,  but  only  about  2  reinforcing 
behaviors  for  comparable  text  space.  Continuing  a  thread,  referring  to  others  by  name,  and  self- 
disclosures  were  the  most  common  indicators  of  social  presence.  They  recommend  that  for  a 
critical  yet  respectful  environment,  e-leaming  must  be  open,  trusting,  close,  warm,  and  personal. 
Yet,  from  their  standpoint,  CMC  groups  must  not  get  too  close,  as  there  still  must  be  a  level  of 
tension  to  foster  cognitive  conflict  and  critical  questions.  Instructional  decisions  concerning  e- 
leaming  definitely  are  not  easy. 
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As  is  evident,  researchers  are  just  beginning  to  ask  questions  and  engage  in  research  related 
to  online  learning  communities.  Many  questions  about  virtual  learning  communities  remain 
(Chao,  1999).  For  instance: 

1 .  What  conditions  foster  online  learning  communities?  What  social  structures  must  be  in 
place? 

2.  At  what  point  does  the  learner  or  participant  become  part  of  the  community? 

3.  How  can  CMC  environments  substitute  for  the  social  cues  of  FTF  environments  that  help 
foster  a  sense  of  community? 

4.  When  and  how  do  students  develop  a  sense  of  online  communities  within  both  training 
and  higher  education  classes?  What  principles,  practices,  and  tools  spur  the  growth  of 
learning  communities? 

5.  How  does  the  development  of  a  learning  community  relate  to  student  perceptions  of 
course  tasks  and  activities? 

6.  Does  the  formation  of  new  relationships  relate  to  the  depth  of  student  learning? 

7.  How  do  instructor  styles,  student  experiences  with  e-leaming,  and  course  materials 
contribute  to  the  development  of  an  online  community? 

8.  How  do  such  characteristics  as  trust,  support,  openness,  knowledge  sharing,  negotiation 
of  meaning,  and  influence  emerge  and  evolve? 

9.  How  do  permanent  learning  communities  differ  from  temporary  classroom-based 
learning  communities? 

10.  Why  do  people  use  a  site?  Why  do  new  people  join  the  asynchronous  learning  network? 
What  motivates  their  participation?  What  are  their  expectations? 

In  addition  to  the  questions  above,  it  is  important  to  understand  the  tools  that  positively 
impact  the  sharing  of  information  and  mutual  understanding  of  participants.  How  do  online  tools 
provide  a  shared  social  space  for  instructor  and  student  interaction?  Just  how  do  participants 
share  knowledge  and  experience?  What  must  be  present  in  the  learning  community  for 
significant  knowledge  negotiation?  As  indicated  by  the  research  studies  suggested  later,  most  of 
these  questions  have  yet  to  be  addressed. 

Other  Variables  of  Interest  . 

In  addition  to  those  variables  mentioned  earlier,  there  is  growing  interest  in  navigational 
strategies,  learning  strategies,  self-efficacy,  self-concept,  and  goal  stability.  For  instance,  in 
examining  verbal  protocols  through  a  hypermedia  database,  Yang  (1997)  created  a  classification 
scheme  for  information  evaluation  and  management  as  well  as  eight  types  of  information 
searching  and  retrieving  behaviors  (e.g.,  prescriptive,  exploratory,  intuitive,  accidental,  curious, 
purposive,  and  tangential).  She  suggests  future  research  explore  how  the  online  task  influences 
the  types  of  learning  strategies  selected  by  learners  and  the  patterns  of  information  searching.  In 
terms  of  collaboration,  Yang  recommends  studies  such  as  comparing  the  learning  strategies  and 
interaction  patterns  of  online  learners  when  collaborating  to  solve  a  problem  versus  collaborating 
to  write  a  report.  For  instance,  researchers  might  document  the  types  of  learning  strategies 
invoked  when  solving  problems— basic  (i.e.,  repeating  information,  copying,  underlining,  etc.) 
or  complex  (e.g.,  paraphrasing,  summarizing,  grouping,  developing  outlines  or  hierarchies,  etc.). 
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Before  such  questions  can  be  answered,  pedagogically  relevant  tools  must  be  designed 
Rich  Lehrer  and  his  colleagues  (Lehrer,  Erickson,  &  Connell,  1994)  argue  that  it  is  important  to 
build  tools  that  engage  students  in  planning,  transforming,  evaluating,  and  revising  their 
knowledge.  Based  on  learner-centered  principles,  Lehrer  (1993)  suggests  that  students  design 
knowledge  instead  of  receiving  information  passively.  Importantly,  he  has  created  a  scheme  to 
analyze  the  cognitive  components  of  hypermedia  design  (defining  problems,  selecting 
information,  organizing  and  representing  information,  evaluating  designs,  and  revising  the 
design).  In  his  HyperAuthor  tool,  students  create  and  label  their  links,  design  graphs  and 
animations,  and  generate  extensive  text.  Such  pedagogical  tools  are  lacking  in  online  learning 
today. 


Research  on  Internet-Based  Distance  Learning 

Distance  learning  refers  to  structured  learning  that  takes  place  without  the  physical 
presence  of  the  instructor.  All  service  branches  are  active  in  the  implementation  of  distance 
learning  technologies  to  replace  traditional  classrooms  for  both  active  and  reserve  components 
(Metzko,  Redding,  &  Fletcher,  1996).  A  recent  report  by  the  Army  Research  Institute 
documented  the  anecdotal  nature  of  research  on  distance  learning  in  training  (Wisher  et  al., 
1999).  Overall,  research  in  distance  learning  often  finds  that  there  are  no  significant  differences 
in  student  learning  and  performance  from  more  conventional  learning  environments.  In  fact, 
there  is  an  online  report  documenting  non  significant  results  of  distance  learning  research  since 
1928  as  well  as  a  scant  few  studies  that  actually  indicate  some  positive  and  negative  differences 
(Russell,  1999,  Appendix  B).  Rather  than  examining  that  literature  any  further,  the  current 
review  will  present  recent  findings  in  education  on  Internet  application  of  e-leaming  and 
collaboration  tools  as  applied  in  distance  learning. 

E-Learning  Tools 

Some  research  reports  are  less  positive  about  e-leaming  than  others.  Hara  and  Kling 
(2000),  for  instance,  received  extensive  press  regarding  their  qualitative  findings  of  student 
anxiety,  frustration,  confusion,  and  lack  of  support  in  an  online  graduate  education  class.  They 
suggest  that  high  quality  e-leaming  may  only  be  possible  when  there  are  highly  dedicated  and 
experienced  teachers  in  this  labor  intensive  business.  Problems  occur  when  student  time  and 
effort  is  not  rewarded  with  feedback,  grades,  or  other  rewards.  There  are  also  a  number  of 
studies  showing  high  rates  of  student  dropout  in  distance  learning  caused,  in  part,  by  the  lack  of 
social  cues,  interaction,  and  clear  expectations  (Abrami  &  Bures,  1996).  Additionally,  some 
studies  show  problems  with  group  leadership  including  domination  and  unresponsiveness  of 
some  online  group  members  (Scrifres,  Gundersen,  &  Behara,  1998).  In  one  study,  library 
science  students  rebelled  against  hypertext-based  instruction  delivered  entirely  over  the  Internet 
(Harris,  Harris,  &  Hannah,  1998).  Even  extremely  competent  and  dominant  students  were 
uncomfortable  with  collaborative  and  electronic  texts.  They  experienced  difficulty, 
disorientation,  disjointedness,  and  confusion  with  the  digital  course.  In  contrast  to  e-leaming 
prophecies,  these  students  were  extremely  open  about  their  love  for  books  and  trust  of  paper,  not 
online  collaboration  tools. 

Some  findings  are  more  mixed.  For  instance,  using  questionnaires,  focus  groups,  and 
structured  interviews,  Small  (1999)  documented  that  simultaneously  working  and  attending 
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college  classes  is  more  difficult  for  resident  than  distance  learning  students,  despite  more  time 
spent  studying  for  distance  education  students.  She  also  found  greater  time  demands  for  faculty 
in  e-leaming  situations.  Among  her  recommendations  were  utilizing  more  flexible  learning 
modes  for  part-time  students  and  using  simple  communication  tools  such  as  email  wherever 
possible.  She  also  suggested  embedding  technologies  that  enhance  student-student  interaction 
and  rapport  with  instructor,  building  experiences  for  interpersonal  bonding,  creating  policies  for 
faculty  time  commitments,  and  having  successful  faculty  model  and  demonstrate  e-leaming 
activities  and  uses  for  others. 

Owston  (1999)  employed  both  quantitative  and  qualitative  tools  in  analyzing  four 
different  online  projects.  Using  structured  interviews,  the  first  study  found  positive  benefits  of 
online  mentoring  within  writing  while  teacher  use  of  a  two-way  television  channel  for  searching 
and  selecting  math  and  science  videos  had  more  limited  success.  Interestingly,  Owston  notes 
that  clustering  network-based  research  projects  that  have  similar  goals  is  more  cost  efficient 
since  evaluators  can  often  make  simultaneous  use  of  the  same  collection  methods,  instruments, 
procedures,  and  staff.  A  second  study  of  online  tutorial  help  in  an  introductory  computer 
technology  course  relied  on  focus  groups,  semi-structured  interviews,  field  observations,  and 
computer  log  files  analyzed  by  WebTrends  Log  Analyzer  software.  Triangulation  of  these  data 
revealed  that  students  did  not  make  extensive  use  of  the  tutorials,  but  those  that  did  found  them 
useful,  easy  to  navigate,  and  motivational.  A  third  study  comparing  students  at  one  college 
taking  courses  via  the  Web,  live  instruction,  or  correspondence,  found  that  the  Web  and  FTF 
courses,  not  surprisingly,  were  superior  to  correspondence  courses  in  terms  of  course 
achievement.  While  cautiously  presented,  the  Web  courses  showed  significant  achievement 
gains  to  FTF  if  students  who  failed  to  complete  the  course  were  removed  from  analyses.  A 
fourth  study  used  transcripts  of  e-mail  and  online  chats,  group  interviews,  and  language 
assessment  tools  to  evaluate  an  English-as-a-second-language  course  for  adults.  While  this 
particular  course  proved  effective,  the  initially  small  enrollment  of  14  students  was  perhaps  not 
as  cost  effective. 

One  recent  report  in  the  Chronicle  of  Higher  Education  (Carr,  2000b;  Appendix  B) 
revealed  that  while  Web  students  in  a  psychology  course  consistently  scored  five  points  higher 
on  final  exams,  they  were  less  satisfied  with  the  course  than  traditionally  taught  students.  The 
researchers  point  out  that  while  students  in  the  FTF  course  tended  to  study  the  night  before  an 
exam,  students  in  the  Internet  course  had  to  space  out  their  studying  practices.  The  e-leaming 
students  also  were  in  an  environment  where  professors  could  respond  to  their  needs  on  demand. 
Similarly,  Davies  and  Mendenhall  (1998)  found  no  significant  differences  between  online  and 
classroom  students  in  a  fitness  and  lifestyle  management  course,  but  most  preferred  a  classroom 
experience.  They  felt  live  instruction  provided  more  entertainment,  friendships,  and  structure 
than  online  experiences. 

In  addition  to  various  negative  or  mixed  e-leaming  findings,  there  are  some  promising 
aspects  to  e-leaming.  In  general,  distance  learners  have  more  positive  attitudes  toward  distance 
learning  than  traditional  learners  (National  Center  for  Education  Statistics,  1999).  An  early 
study  of  online  environments  by  Hiltz  (1990)  also  revealed  some  advantages  of  online  over 
traditional  such  as  greater  access  to  professors,  increases  in  participation,  higher  course 
satisfaction,  equal  or  superior  mastery  of  course  material,  and  enhanced  interest  in  subject 
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matter.  Hiltz  admitted  that  such  gains  rely  on  having  motivated  and  prepared  students  with 
adequate  access  to  technology.  More  recent  findings  do  confirm  that  distance  learning  students 
tend  to  be  more  intrinsically  motivated  and  internally  controlled  (National  Center  for  Education 
Statistics,  1999). 

More  recently,  Stefl-Mabry  (1998)  found  an  increase  in  reading  comprehension  for 
college  students  in  a  Web-based  “Introduction  to  Reading”  course  over  students  in  a  traditional 
version  of  the  course.  In  college  economics  courses,  Agarwal  and  Day  (1998)  discovered  that 
Internet  enhancements  raised  student  attitudes,  performance  on  exams  and  course  grades,  and 
student  attitudes  toward  economics.  In  another  study  of  college  students,  students  who  were 
active  in  an  optional  computer-mediated  communication  (CMC)  exercise  also  received  higher 
course  grades  (Althaus,  1997).  However,  it  was  unclear  whether  this  was  truly  due  to  learning  in 
the  course  or  preexisting  differences. 

Collaborative  Tools  in  Distance  Learning 

There  are  both  positive  and  negative  effects  related  to  collaborative  tools.  For  instance, 
even  though  students  can  participate  at  any  time,  this  technological  advantage  also  places  more 
requirements  on  the  instructor  to  be  responsive  at  nearly  any  time  (Ottenhoff  &  Lawrence,  1999). 
Whereas  online  discussion  forums  can  provide  unique  venues  for  public  reflection,  they  can  also 
become  filled  with  careless  and  thoughtless  entries.  Along  these  same  lines,  online  forums  can 
be  opportunities  for  students  to  debate  and  negotiate  arguments  or  a  place  wherein  the  instructor 
has  the  ultimate  final  word.  In  effect,  online  collaboration  tools  can  be  utilized  for  significant 
change  in  adapting  student-centered  approaches  or  they  can  continue  fostering  a  more  passive 
learning  approach.  Clearly,  the  technologies  alone  will  not  change  the  educational  process. 
Innovative,  planful  curricula  that  structures  online  interaction  for  meaningful  and  authentic 
learning  experiences  increases  the  chance  for  successful,  student-centered  learning  (Riel,  1990). 

Merisotis  (1999)  points  out  that  since  distance  learning  technology  will  continue  to  play  a 
significant  role  in  teaching  and  learning,  the  focus  should  shift  from  whether  it  makes  a 
difference,  to  where  it  makes  a  difference  as  well  as  which  approaches  might  have  more 
powerful  impacts  in  terms  of  student  learning.  Stephen  Ehrmann  (1999,  p.  51)  adds  that  “the 
real  debate  needs  to  focus  on  identifying  which  approaches  work  best  for  teaching  students, 
period.”  Of  course,  a  focus  on  learner-centered  instruction  and  pedagogy  might  create  more 
avenues  for  new  e-leaming  technologies  to  be  deemed  effective.  Even  the  findings  of  extensive 
cost  and  performance  analyses  of  large-section  college  classes  taught  online  such  as  those  at  the 
Sloan  Center  for  Asynchronous  Learning  Environments  (SCALE)  at  the  University  of  Illinois 
can  be  reduced  to  one  common  finding — online  courses  need  sensible  pedagogical  approaches 
that  allow  students  opportunities  to  communicate  their  learning  (Arvan,  Ory,  Bullock,  Bumaska, 
&  Hanson,  1998). 

Educational  Websites 

Just  as  the  debates  about  the  impact  of  distance  education  on  student  access  and  learning 
are  inconclusive,  so  too  are  the  reviews  of  educational  Websites.  As  noted  earlier,  instructors 
tend  to  employ  simple  tools  online.  A  study  of 436  educational  Websites  by  Mioduser, 
Nachmias,  Lahav,  and  Oren  (1998)  demonstrated  that  instructors  and  students  utilize  extremely 
limited  communication  and  interaction  tools,  such  as  e-mail.  While  simple  tools  can  help  ease 
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the  frustrations  and  tensions  faced  online,  they  typically  do  not  offer  many  opportunities  to 
negotiate  meaning  and  counter  ideas.  In  the  Mioduser  et  al.  study,  tools  for  facilitating  work 
groups,  learning  teams,  and  learning  communities  were  virtually  nonexistent.  Such  findings  are 
epitomized  in  their  conclusions  that  for  every  “one  step  ahead  for  technology,”  there  are  “two 
steps  back  for  the  pedagogy.” 

In  reviewing  college  syllabi  posted  at  the  World  Lecture  Hall  (Appendix  B),  Cummings 
et  al.  (2000)  similarly  found  that  few  faculty  utilized  the  numerous  interaction  and  collaborative 
possibilities  of  the  Web.  For  instance,  few  instructors  embedded  opportunities  for  practitioner 
and  guest  expert  interaction.  Instead,  most  syllabi  were  marked  by  one-way  flows  of  information 
from  instructors  to  their  students.  Yet,  according  to  research  by  Hiltz  (1998),  collaborative 
learning  and  interaction  are  vital  to  the  success  of  asynchronous  learning  networks.  She 
concludes  that  “Colleges  and  universities  ought  to  be  concerned  not  with  how  fast  they  can  ‘put 
their  courses  up  on  the  Web,’  but  with  finding  out  how  this  technology  can  be  used  to  build  and 
sustain  learning  communities”  (p.  7).  In  addition  to  fostering  online  course  sharing  and 
collaboration,  Web  tools  have  the  potential  to  promote  new  partnerships  and  build  efficiencies  in 
course  offerings  (Page,  1999). 

There  is  little  indication  that  such  debates  will  subside  soon,  but  one  thing  is  certain— 

Web  technologies  can  be  used  in  profound  as  well  as  extremely  trivial  and  careless  ways 
(Ottenhoff  &  Lawrence,  1999).  Those  researching  collaborative  learning  tools  and  online  social 
interaction,  both  in  training  and  education  environments,  are  making  serious  attempts  to  find  the 
more  profound  and  meaningful  uses. 

Survey  and  Evaluation  Tools 

One  common  evaluation  tool  in  distance  learning  is  the  student  and  instructor  survey.  In 
brief  training  sessions,  a  short  self-evaluation  can  offer  the  most  cost  effective  and  informative 
tool  (Wisher  &  Cumow,  1998).  Such  tools  might  foster  dynamic  changes  in  instruction.  Self- 
evaluations  are  faster,  less  costly,  more  convenient,  and  offer  comparable  data  to  more  objective 
measures,  particularly  in  military  training  environments  (Cumow  &  Wisher,  2000).  In  corporate 
training  environments,  evaluation  often  focuses  on  practical  data  such  as  product  surveys, 
usability  analyses,  hands-on  daily  usage  logs,  and  pilot  tests  (Sokolov,  1999).  Some  researchers 
have  used  Likert  scale  surveys  of  student  course  satisfaction  to  evaluate  online  courses.  For 
instance,  Schlough  and  Bhuripanyo  (1998)  examined  the  clarity,  relevance,  accuracy, 
organization,  and  sequence  of  content  as  well  as  the  effectiveness  or  graphics  and  design  in 
meeting  student  needs. 

In  evaluating  two  technologically  delivered  graduate  programs  at  Nova  Southeastern 
University,  Hessler  and  Kontos  (1996-97)  surveyed  the  technological  support  provided  by  the 
campus,  preexisting  student  computing  backgrounds,  and  their  ability  to  use  and  participate  in 
different  media  in  the  program  such  as  the  electronic  classroom,  online  research  tools,  and 
electronic  mail.  In  using  conceptual  framework  for  evaluating  for  distance  education  teaching, 
Cheung  (1998)  developed  a  35  item  questionnaire  involving  (1)  student  development,  (2) 
assessment,  (3)  learning  materials,  and  (4)  FTF  components.  Similarly,  a  25  question  distance 
education  evaluation  device  from  Spooner,  Jordan,  Algozzine,  and  Spooner  (1999)  examined  the 
course,  instructor,  organization,  teaching,  communication,  and  the  overall  experience. 
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Such  surveys  indicate  that  positive  learner  perceptions  of  interaction  are  critical  for  e- 
leaming  success  (Fulford  &  Zhang,  1993)  as  are  self-motivation  and  self-discipline  (Gifford, 

1 998).  Other  research  reveals  that  students  feel  that  online  courses  take  more  time  than 
conventional  courses  but  that  detailed  timelines  and  external  support  such  as  imposed  deadlines 
help  them  complete  these  courses  and  tasks  (Mory,  Gambill,  &  Browning,  1998).  Not 
surprisingly,  students  in  control  conditions  without  exposure  to  such  technologies  evaluate  the 
potential  of  e-leaming  environments  less  favorably  (Usip  &  Bee,  1998).  As  the  e-leaming 
education  and  training  continues  to  grow,  there  will  be  even  greater  need  for  standard  and 
adaptable  online  surveys.  The  Internet  can  offer  quick  turnaround  and  reduced  costs  for 
conducting  evaluations  (Kronholm,  Wisher,  Cumow,  &  Poker,  1999). 

Affective  constructs.  In  addition  to  cognitive  strategies,  distance  learning  impacts  many 
affective  constructs.  For  instance,  Hill  and  Hannafin  (1997)  hypothesized  that  students  with 
higher  self-efficacy  will  not  only  be  more  confident  in  using  the  Web,  but  they  will  take  more 
risks.  Using  subject  surveys  and  simulated  post  search  interviews,  their  research  indicated  that 
students  with  higher  self-efficacy  engaged  in  more  strategic  behaviors  and  at  higher  levels  than 
low  self-efficacy  participants.  Such  students  also  were  more  exploratory  and  felt  more  in  control 
of  their  learning.  These  researchers  argued  that  metacognitive  knowledge,  perceived  orientation 
or  awareness  of  location  within  the  system,  system  knowledge,  and  prior  knowledge  of  the 
subject  also  influenced  success  in  learner-centered,  open-ended  environments. 

While  a  popular  study  found  that  greater  Internet  use  lowered  participation  in  family 
activities  and  communication  while  increasing  feelings  of  loneliness  and  depression  (Kraut, 
Patterson,  Lundmark,  Kiesler,  Mukopadhyay,  &  Scherlis,  1998),  it  is  important  to  ask  whether 
increased  distance  education  course  experience  harms  student  psychological  well-being.  In 
addition,  how  does  such  e-leaming  coursework  impact  academic  self-concept?  Research  from 
Gibson  (1996)  indicates  that  academic  self-concept  seems  to  play  a  significant  role  in  student 
persistence  and  ultimate  success  in  distance  education.  According  to  her  work,  clear  instructor 
expectations,  ready  access  to  faculty,  control  over  learning,  and  other  support  mechanisms 
positively  influence  student  academic  self-concept  in  distance  education  environments.  From 
such  research,  it  is  clear  that  external  support  mechanisms  seem  to  be  crucial  to  e-leaming 
success. 

Researchers  are  beginning  to  ask  whether  such  support  must  come  from  instructors  or  if 
peer  support  can  be  as  effective?  In  comparing  instructor-student  e-mail  feedback,  system- 
student  feedback,  and  student-student  online  feedback,  Rada  (1998)  found  that  students  were 
more  satisfied  with  the  distance  learning  experience  when  peers  or  instructors  responded  to  their 
work  than  when  a  database  management  system  provided  the  feedback.  In  this  study,  student- 
student  interaction,  instructor  monitoring,  and  quality  control  tended  to  produce  the  most 
favorable  student  evaluations.  Since  peer  feedback  was  as  effective  as  that  from  the  instructor, 
he  concluded  that  the  instructor  need  not  be  the  only  form  of  online  feedback. 

While  this  section  details  a  number  of  recent  e-leaming  studies,  it  is  far  from  exhaustive. 
It  is  also  important  to  describe  research  related  to  developing  online  communities. 
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Research  Gaps 

There  is  a  pressing  need  for  more  research  on  e-leaming  environments  (The  Report  of  the 
University  of  Illinois  Teaching  at  an  Internet  Distance  Seminar,  1999).  Davenport  and  McKim 
(1995)  note  the  dearth  of  qualitative  research  and  naturalistic  studies  of  groupware  and 
collaborative  tools.  A  combination  of  qualitative  and  quantitative  research,  however,  may  help 
understand  the  mixed  results  that  are  far  too  common  in  comparative  studies  of  distance  learning 
(Scrifres  et  al.,  1998)  while  identifying  areas  of  impact,  instead  of  providing  yet  another  study 
with  no  significant  differences.  There  is  a  shift  from  focusing  on  how  individuals  function  in 
different  types  of  groups  and  within  different  activities,  to  targeting  the  group  itself  as  the  unit  of 
analysis  (Dillenbourg,  Baker,  Blaye,  &  O’Malley,  1996).  What  types  of  interactions, 
explorations,  negotiations,  and  explanations  within  the  group  impact  individual  and  group 
performance?  Key  questions  remain  related  to  how  students  construct  and  negotiate  meaning  in 
online  collaborative  environments  (Warschauer,  1997)  as  well  as  the  role  of  the  instructor  in 
such  environments.  Experiments  designed  to  address  some  of  these  issues  is  provided  below. 


Ten  Primary  Experiments 

The  educational  literature  has  been  reviewed.  Based  on  the  findings  relevant  to  the 
Army’s  transformation  to  a  soldier-centric  learning  model,  the  following  ten  experiments  form  a 
research  framework  for  adapting  educational  tools  to  training  environments.  They  are  not  listed 
in  order  of  importance  nor  are  they  suggestive  of  the  order  in  which  they  might  be  conducted. 
Each  experiment  is  first  described  by  a  narrative  description  that  extracts  findings  from  the 
literature  and  then  in  a  tabular  form  identifies  the  key  independent  and  dependent  variables  for 
testing  in  an  Army  setting.  Each  experiment  can  be  conducted  in  a  future  Army  distance 
learning  program  or  a  similar  program  in  another  service.  Depending  on  the  specifics  of  the 
program  and  e-leaming  environment  under  investigation,  the  variables  might  need  to  be 
extended,  reduced,  or  otherwise  modified  to  suit  a  particular  training  environment.  Any  of  the 
experiments  suggested  here  must  of  course  be  further  developed  with  a  complete  research  or 
evaluation  plan. 

As  indicated  earlier,  much  of  the  research  evidence  for  the  training  effectiveness  of 
distance  learning  has  been  criticized  as  being  anecdotal,  poorly  controlled,  and  not  supported  by 
adequate  experimental  designs  (Wisher  &  Champagne,  2000).  Such  practices  leave  too  much 
room  for  potential  threats  to  the  internal  validity  of  the  study.  For  example,  some  studies  do  not 
take  into  account  a  mortality  effect  in  which  students  with  less  ability,  motivation,  or  time  may 
become  discouraged  and  drop  out  during  the  program  so  that  the  average  posttest  knowledge 
scores  are  higher  than  the  average  pretest  scores  for  reasons  other  than  the  treatment.  As 
identified  in  Champagne  and  Wisher  (2000),  experimental  design  practices  that  include 
comparison  groups,  random  assignment  of  students,  and  multiple  outcome  measures  should  be 
included  wherever  possible.  The  experiments  described  below  are  those  that  can  most 
immediately  guide  the  Army  in  adapting  promising  e-leaming  technologies  to  training 
environments. 

Criteria  for  developing  the  experiments  included  an  implicit  assumption  that  certain  tools 
demonstrated  successfully  in  education  applications  are  likely  to  be  of  benefit  in  military 
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training.  Proposed  experiments  are  designed  to  foster  near-term  technology  application  and  not 
basic  research.  The  research  strategy,  then,  is  to  design  research  to  satisfy  Army  schools 
requiring  or  planning  collaborative  learning  environments,  and  to  develop  experiments  that 
advance  the  development  and  adaptation  of  educational  technologies  to  meet  those  Army  needs. 
The  initial  candidates  for  application  are  the  Armor  Captains  Career  Course  (CCC)  and  the  Field 
Artillery  CCC.  Several  research  efforts  should  directly  support  collaborative  learning  in  distance 
learning  versions  of  these  courses.  The  first  experiment  among  the  ten  proposed  below  has  the 
highest  potential  payoff.  The  others,  while  interesting,  have  no  obvious  near-term  Army  payoff. 

Experiment  1.  Online  Discussion:  Online  Officer  Training  Program 

Cummings  (2000)  notes  that  online  discussions  allow  participants  to  reread  statements, 
prepare  positions  and  rebuttals,  exchange  ideas,  and  interact  with  peers,  while  providing  a 
permanent  record  of  the  discussion.  In  this  experiment,  students  in  an  online  training  program, 
such  as  the  first  or  second  phases  of  a  Captains  Career  Course,  will  be  assigned  to  one  of  four 
groups  during  the  asynchronous  component  of  the  course:  (1)  discussion  with  preassigned  roles 
with  equal  pro  and  con  positions;  (2)  discussion  with  student  selected  roles;  (3)  discussion  with  a 
few  assigned  controversial  roles  (e.g.,  devil’s  advocate,  pessimist,  etc.);  or  (4)  discussion  with  no 
roles.  A  control  group  should  also  be  used.  Depending  on  the  size  of  the  class,  there  may  be 
subgroups  within  each  treatment.  Variables  explored  will  include  how  the  form  of  discussion 
impacts  the  depth  of  student  discussion  as  measured  through  content  analysis  techniques.  Will 
students  simply  share  personal  ideas  or  negotiate  ideas  and  eventually  compromise  on  positions? 
The  main  hypothesis  is  that  specifically  assigning  student  roles  that  foster  cognitive  conflict,  as 
in  Groups  1  and  3,  will  foster  more  social  interaction  and  negotiation  of  meaning.  These  groups 
will  also  learn  the  most.  A  second  hypothesis  is  that  self-regulated  learners  will  more 
extensively  participate  in  the  discussion;  especially  in  Group  2.  Third,  students  in  Group  2  will 
also  have  the  highest  attitudes  about  the  discussion  activity.  A  fourth  hypothesis  is  that  Group  4 
discussions  will  be  at  the  surface  level  and  will  quickly  die  out.  An  alternative  or  follow-up 
experiment  would  have  two  forms  of  discussion  (assigned  and  unassigned  roles)  and  two  forms 
of  communication  (synchronous  and  asynchronous).  The  first  hypothesis  here  is  that  students  in 
the  synchronous  discussion  with  assigned  roles  will  likely  be  more  task  focused  than  students  in 
the  real-time  environment  without  roles.  Second,  students  in  both  synchronous  environments 
may  form  relationships  more  quickly  than  students  in  the  asynchronous  environments  (Walther, 
1992).  Third,  the  synchronous  discussion  will  have  more  ideas,  whereas  the  asynchronous 
discussion  will  have  more  reflective,  integrative,  and  evaluative  comments. 


Initial  experiment:  Asynchronous  communication  in  assigned  roles. 


Independent  variable:  Group  membership 
(5  levels). 

Dependent  variable:  Impact  of  discussion 
form  on  depth  of  student  discussion. 

1.  Discuss  with  pre-assigned  roles 
with  equal  pro  and  con  positions. 

2.  Discuss  with  student  selected  roles. 

3.  Discuss  with  a  few  assigned 
controversial  roles  (e.g.  devil’s 
advocate). 

4.  Discuss  with  no  roles. 

5.  Control  (no  discussion). 

•  Degree  to  which  students  simply 
share  personal  ideas? 

•  Degree  to  which  students  negotiate 
and  eventually  compromise? 

•  Depth  of  learning  as  measured  by 
content  analysis. 
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Hypotheses: 

1 .  Specifically  assigning  students  roles  that  foster  cognitive  conflict  (Group  1  and  3)  will  foster 
more  social  interaction  and  negotiating  of  meaning. 

2.  Groups  1  and  3  will  learn  the  most. 

3.  Self-regulated  learners  will  participate  in  the  discussion  more  extensively,  especially  in  Group  2. 

4.  Group  2  will  have  the  strongest  attitudes  toward  the  discussion  activity. 

5.  Group  4  discussions  will  be  at  the  surface  level  and  will  quickly  die  out. 

Follow  up  experiment:  Synchronous  and  asynchronous  communication  in  both  assigned  and 


unassigned  roles. 


Independent  variables: 

Dependent  variables: 

A.  Form  of  communication  (2  levels). 

1 .  Synchronous. 

2.  Asynchronous. 

B.  Form  of  discussion  (2  levels). 

1 .  Assigned  roles. 

2.  Unassigned  roles. 

A.  Degree  of  task  focus. 

B.  Speed  of  relationship  formation. 

C.  Depth  of  communication. 

D.  Transfer  to  a  measurable  learning 
outcome. 

Hypotheses: 

1 .  The  synchronous  discussion  group  with  assigned  roles  will  be  more  task  focused  than  the 
synchronous  group  with  unassigned  roles. 

2.  The  subjects  in  both  synchronous  groups  (assigned  and  unassigned)  will  form  relationships 
more  quickly  than  subjects  in  the  asynchronous  groups. 

3.  The  synchronous  discussion  groups  will  generate  more  ideas,  whereas  the  asynchronous 
groups  will  be  more  reflective,  integrative  and  evaluative. 

Theory: 

Cummings  (2000)  notes  that  online  discussions  allow  participants  to  reread  statements, 
prepare  positions  and  rebuttals,  and  interact  and  exchange  ideas  with  peers,  while  lowering 
anxiety  about  peer  positions  and  providing  a  permanent  record  of  the  discussion. 

Walther  (1992)  found,  that  synchronous  environments  foster  quicker  relationship  formation  than 
asynchronous  environments. 

Experiment  II.  Variations  in  Instructor  Moderation:  E-Learning  Environments 

The  instructor  plays  a  critical  role  in  online  training  environments.  Research  to  date 
indicates  that  in  e-leaming  environments,  instructors  need  to  shift  from  traditional  lecture-based 
approaches  to  being  facilitators  of  learning.  But  how  do  instructors  foster  student  social 
interaction  and  knowledge  construction  online?  How  can  instructors  raise  the  level  of  online 
questioning  and  interaction?  How  can  instructors  help  move  learners  from  being  information 
receptors  to  actively  interpret,  question,  challenge,  inquire,  create,  discuss,  and  negotiate  ideas? 
What  types  of  online  supports  are  needed  to  move  learners  from  simply  assimilating  archived 
knowledge  to  places  wherein  students  personally  construct  their  own  personal  and  group 
knowledge  (Dede,  1996b)?  A  survey  tool  on  teaching  metaphors  (i.e.,  instructor  as  coach, 
expedition  leader,  etc.)  will  be  developed  and  used  to  determine  the  type  of  instructional 
approaches  adopted  online.  Instructors  in  five  classes  will  be  provided  with  training  materials  on 
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different  types  of  online  teaching  approaches:  (1)  instructor  as  guide;  (2)  instructor  as  host  or 
friend;  (3)  instructor  as  provocateur;  (4)  instructor  as  teacher  or  knowledge  provider;  and  (5)  a 
combination  approach.  Online  transcripts  will  be  coded  to  determine  the  type  of  instructional 
style  employed  online  and  then  compared  to  the  training.  Just  what  forms  of  instructional 
scaffolding  and  support  do  instructors  with  each  style  tend  to  use?  Techniques  to  facilitate 
student  collaboration  will  be  emphasized.  Students  from  each  class  will  be  asked  to  complete  a 
questionnaire  on  their  perceptions  of  the  instructional  approaches  used  during  their  course  that 
will  be  compared  to  the  intended  treatment.  Student  preferences  for  e-leaming  and  sense  of 
leamer-centeredness  will  be  compared  to  the  instructional  approach  of  that  class.  A  follow-up  or 
alternative  experiment  would  expand  the  teaching  metaphors  to  20-30  options  as  well  as  the 
number  of  instructors  involved  and  have  them  select  the  ones  that  best  describe  their  online 
teaching.  Such  a  survey  could  determine  not  only  the  type  of  instructional  approach  best  suited 
to  e-leaming,  but  also  ferret  out  any  differences  in  the  instructor  role  in  simple  and  more 
complex  e-leaming  courses.  An  additional  study  might  entail  using  such  a  survey  over  a  number 
of  weeks  or  months  to  note  any  changes  in  instructor  perceptions  of  their  role  over  time.  The 
relationship  between  student  performance  and  learning  outcome  will  be  examined. 


Initi al  experiment:  Instructional  approach  preferences. 


Independent  Variable:  Online  teaching 
approach  (5) 

Dependent  variable:  Student  preferences 

1.  Teacher  as  guide 

2.  Teacher  as  host,  or  friend 

3.  Teacher  as  provocateur 

4.  Teacher  as  instructor 

5.  Combination 

•  Preference  for  e-leaming 

•  Sense  of  leamer-centeredness 

Follow  up  survey:  Determine  best  approach  for  e-leaming  (depending  on  the  complexity  of  the 
course). 

•  Present  20-30  teaching  metaphors  (i.e.,  teacher  as  a  coach,  expedition  leader,  etc.) 

•  Have  instructors  select  the  metaphors  that  best  describe  their  online  teaching  approach. 

•  Look  for  differences  in  the  role  of  the  instructor  for  complex  and  simple  courses. 

Experiment  III.  Student  Perceptions  of  E-Learning  Environment 

In  the  training  literature,  student  perception  of  the  overall  learning  environment  has  been 
an  uncertain  predictor  of  student  achievement  whereas  student  perception  of  interactivity 
corresponds  to  learning  achievement  (Payne,  1 999).  There  is  a  more  direct  relationship  between 
overall  perception  and  achievement  in  educational  settings.  The  issues  in  this  experiment  are:  Do 
students  perceive  the  Web  as  an  individual  learning  tool,  a  competitive  learning  environment,  or 
a  place  for  collaboration?  When  students  view  a  collaborative  tool  as  useful  only  for  completing 
their  individual  work,  then  it  is  problematic.  If  the  intended  tool  does  not  match  perceived  utility 
on  the  part  of  the  learner,  then  hypotheses  in  many  distance  education  environments  will  be 
faulty.  In  this  experiment,  the  variable  of  interest  is  student  perceptions  of  their  learning 
environment.  A  survey  instrument  will  be  created  that  explores  whether  students  in  an  online 
training  environment  embedded  with  extensive  collaborative  tools  perceive  an  emphasis  on 
individual,  competitive,  or  collaborative  learning.  In  addition,  interviews  with  students  will 
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determine  the  types  of  activities  that  are  truly  collaborative  and  the  associated  tool  features  that 
enhance  feelings  of  collaboration.  Electronic  transcripts  of  tasks  performed  online  will  be  coded 
for  collaborative,  competitive,  and  individual  learning  acts.  The  transcripts  will  also  be  utilized 
in  student  interviews  during  retrospective  analyses  of  learning.  Students’  perceptions  of  the 
actual  type  of  learning  environment  will  be  compared  with  completed  surveys  of  preferred 
learning  environments.  Student  perceptions  of  their  learning  environment  will  be  correlated  with 
their  actual  performance,  thereby  providing  one  indicator  of  the  type  of  e-leaming  environment 
that  might  foster  student  learning.  Comparisons  of  student  perceptions  in  procedural  and  more 
complex  learning  environments  (e.g.,  officer  training)  will  indicate  whether  different  types  of 
courses  force  different  pedagogical  approaches. 


Independent  variable  (predictor):  Student’s 
perception  of  emphasis  of  online  training  (3). 

Dependent  variable  (criterion): 

Performance 

1 .  Individual  learning 

2.  Competitive  learning 

3.  Collaborative  learning 

•  Actual  course  performance 

Follow  up  study: 

•  Interview  students  to  determine  the  types  of  activities  and  associated  tools  they  consider 
collaborative. 

•  Code  electronic  transcripts  of  online  tasks  for  collaborative,  competitive  and  individual  acts. 

•  Use  transcripts  for  future  interviews  and  analysis  of  student  learning. 

Compare  student  perceptions  across  task  complexity  to  determine  if  different  courses  force 
different  pedagogical  approaches. 


Experiment  IV.  The  Development  of  Online  Learning  Communities 

Interaction  is  central  to  learning.  Tools  to  foster  student  social  interaction  and 
collaboration  are  proliferating  in  distance  education  environments.  As  a  result,  many  researchers 
are  focusing  on  the  development  of  online  learning  communities.  Military  recruits  taking  a 
series  of  e-leaming  courses  are  involved  in  many  new  types  of  learning  communities.  How  do 
they  react  to  the  different  types  of  courses?  How  are  electronic  communities  formed?  What  are 
the  key  components?  What  factors  help  with  student  retention  in  those  courses?  And  what  are 
some  early  indicators  of  successful  online  learning  communities?  In  this  study,  the  Social 
Constructivism  and  Learning  Communities  Scale  Online  (SCALCO)  student  survey  will  be 
developed  and  tested  (see  Table  12,  Appendix  A).  The  SCALCO  will  explore  the  degree  of 
social  interaction  and  constructivism  fostered  by  online  collaborative  tools  as  well  as  the  factors 
leading  to  successful  online  communities.  It  will  be  administered  to  students  at  the  start  and  end 
of  10-15  online  courses.  The  results  from  a  content  analysis  of  the  tools  and  activities  embedded 
in  these  courses  will  be  compared  to  student  answers.  Interaction  analyses  of  electronic 
transcripts  will  determine  the  degree  of  interactivity  and  responsiveness  in  online  collaboration. 
An  instructor  version  of  the  SCALCO  will  also  be  created  in  order  to  compare  student  and 
instructor  perceptions  of  online  learning  environments.  Researchers  will  interview  at  least  six 
students  and  four  instructors  to  determine  components  leading  to  online  communities. 
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Study  Purpose:  Use  SCALCO  to  explore  the  degree  of  social  interaction  and  collaboration 
fostered  by  online  collaborative  tools,  and  the  factors  leading  to  successful  online  communities. 
Pre  Post  test  design _ _ 

•  Administer  SCALCO 

•  Expose  subject  to  10-15  online  courses 

•  Administer  SCALCO 

•  Content  analysis  of  the  tools  and  activities  embedded  in  these  courses 

•  Compare  content  to  student  answers 

•  Interaction  analysis  of  electronic  transcripts  (determine  the  degree  of  interaction  and 

responsiveness) _ _ 


Follow  up: 

•  Develop  instructor  version  of  SCALCO 

•  Administer  SCALCO  to  instructors 

•  Compare  student  and  instructor  perceptions  of  online  learning  environments 

•  Interview  at  least  six  students  and  four  instructors  to  determine  components  leading  to  online 
communities 

Experiment  V.  Time  Logging:  Effect  of  E-Learning  Approach  on  Time 

Most  instructors  and  students  find  that  e-leaming  courses  are  highly  time  intensive 
compared  to  traditional  instruction.  Thus,  time  is  a  key  factor  in  the  success  of  e-leaming.  In 
this  study,  four  instructors  will  log  their  time  spent  interacting  in  an  online  course  over  a  three- 
month  period.  After  a  brief  training  program,  each  instructor  will  assume  a  different  pedagogical 
strategy:  (1)  guide  or  facilitator  of  student  learning  (i.e.,  using  indirect  or  Socratic  teaching 
methods);  (2)  lecturer  or  expert  mentor  (i.e.,  direct  instructional  approaches);  (3)  co-participator 
or  co-leamer  (i.e.,  learning  and  discovering  as  a  fellow  students);  or  (4)  host  or  hostess  (i.e., 
coordinating  events,  managing  activities,  and  helping  students  meet  each  other).  Time  spent  by 
the  instructor  will  be  compared  to  the  particular  form  of  teaching  employed.  The  hypothesis  here 
is  that  the  instructional  approach  will  significantly  impact  on  the  amount  of  instructor  time. 
Students  in  these  classes  also  will  be  asked  monthly  to  record  their  time,  both  in  total  number  of 
hours  per  day  or  week  as  well  as  timing.  Student  performance  will  be  compared  to  whether  they 
adopted  a  massed  or  distributed  practice  approach  for  completing  activities  and  studying  for 
exams.  Their  time  logs  will  also  be  compared  to  the  instructional  format  for  indicators  of  the 
instructional  approach  that  fostered  greater  student  participation  and  engagement.  The  key 
hypothesis  here  is  that  when  instructors  assume  a  lecturer  role  (Condition  #2),  student  online 
participation  decreases. 


Experiment:  Instructor  will  log  time  spent  on  course  over  a  three-month  period. 


Independent  variable:  Pedagogical  strategy  (4) 

Dependent  variable:  Time  spent 

1 .  Guide  or  facilitator  (indirect) 

2.  Lecturer  or  expert  mentor  (direct) 

3.  Co-participator  or  co-leamer 

4.  Host  or  coordinator 

•  As  recorded  in  instructor’s  log 

Hypothesis: 

1 .  Instructor  approach  will  be  significantly  related  to  the  amount  of  time  spent  by  the  instructor. 
Follow  up: 
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•  Students  will  log  time  spent  (monthly). 

•  Students  will  indicate  whether  they  used  massed  or  distributive  practices  when  completing 
tasks  and  studying  (IV). 

•  Compare  practice  style  to  performance  (DV). 

•  Compare  student  time  logs  to  type  of  instructional  approach. 

Hypothesis: 

1 .  Lecture  approach  (Condition  2)  will  decrease  student  online  participation. 

Experiment  VI.  Critical  Thinking  and  Problem  Solving  Applications  in  Synchronous  and 
Asynchronous  Environments 

The  development  of  critical  thinking  and  problem  solving  skills  is  central  to  learning. 
Garrison  (1991)  mapped  out  five  key  stages  of  adult  critical  thinking  and  problem-solving  that 
are  often  used  to  analyze  the  effectiveness  of  e-leaming  environments.  But  do  online 
environments  really  impact  student  problem  solving  abilities?  Using  Newman,  Johnson, 
Cochrane,  and  Webb’s  (1996)  model  for  exploring  critical  thinking  in  online  discourse,  students 
in  one  military  training  course  will  be  divided  into  four  collaborative  groups  (see  Table  7).  The 
first  group  will  utilize  synchronous  chat  tools  to  generate  problem  statements  and  ideas  for 
solution,  followed  by  asynchronous  tools  for  idea  evaluation  and  integration.  The  second  group 
will  engage  in  the  reverse  situation  of  Group  1 — asynchronous  generation  of  ideas  followed  by 
synchronous  evaluation,  ranking,  and  integration.  Group  3  will  utilize  synchronous  tools  for 
both  problem  generation  and  evaluation.  Finally,  the  last  group  will  use  only  asynchronous 
tools.  Student  attitudes  about  each  environment  will  be  collected  and  compared.  Questions  to  be 
asked  include:  What  is  the  level  of  involvement  across  different  group  members  during  different 
phases  of  a  task?  What  tools  will  foster  greater  learning  and  interaction  at  different  problem 
solving  stages?  How  successful  is  networking  in  terms  of  facilitating  appropriate  levels  of 
interaction  among  the  participants?  What  phases  or  steps  are  involved  in  the  definition  and 
completion  of  shared  tasks?  Transcripts  will  be  coded  using  Newman  et  al.’s  scheme  and 
compared  for  the  types  of  critical  thinking  embedded  in  the  dialogue  (e.g.,  justification,  idea 
linking,  critical  assessment,  relevancy,  importance,  etc.).  In  addition,  the  Constructivist 
Interaction  Analysis  Coding  Scheme  of  Kanuka  and  Anderson  (1998;  see  Table  8,  Appendix  A) 
will  be  utilized  to  determine  the  phase  of  social  construction  of  knowledge  that  each  group 
obtained.  Next,  the  online  dialogue  will  be  coded  for  the  type  of  utterances  in  online 
collaboration  (Curtis  &  Lawson,  1999,  see  Table  9).  To  what  degree  were  students  in  each 
condition  planning,  contributing,  seeking  input,  reflecting  or  monitoring  on  their  work,  and 
socially  interacting?  What  help  giving  behaviors  positively  impact  on  performance? 
Participation  will  be  coded  for  surface  or  in-depth  commenting  to  determine  which  forum  fosters 
more  integrative  and  deep  thinking  (see  Table  6).  Finally,  changes  in  interaction  patterns  will  be 
mapped  out  over  time.  A  key  hypothesis  here  is  that  Group  2  (synchronous  followed  by 
asynchronous  tools)  will  be  the  most  productive  in  terms  of  number  of  ideas  generated  and 
evaluated.  This  group  will  exhibit  higher  levels  of  social  construction  of  knowledge  than  the 
other  groups  since  the  real-time  chat  will  allow  group  members  to  learn  about  each  other  and 
generate  many  ideas  while  the  follow-up  delayed  discussions  will  promote  negotiation  of  those 
ideas. 
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Experiment:  Effect  of  e-leaming  tools  on  learning  and  interaction. 


Independent  variable:  Course 
characteristics  (4) 

Dependent  variable:  Student  attitudes  and 
perfomance 

1 .  Synchronous  generation  of  ideas, 
asynchronous  evaluation 

2.  Asynchronous  generation  of  ideas, 
synchronous  evaluation 

3.  Synchronous  tools  for  both  generation 
of  ideas  and  evaluation 

4.  Asynchronous  tools  for  both  generation 
of  ideas  and  evaluation 

•  Level  of  involvement  across  group 
members  and  tasks 

•  Which  tool  characteristics  will  foster 
the  greatest  learning  and  interaction? 

•  Number  of  ideas  generated? 

•  Number  of  ideas  evaluated? 

Follow  up: 

•  Code  transcripts  using  Newman  et  al.’s  (1996)  model  for  exploring  critical  thinking  in  online 
discourse  (e.g.,  justification,  idea  thinking,  critical  assessment,  etc.). 

•  Use  the  Constructivist  Interaction  Analysis  Coding  Scheme  of  Kanuka  and  Anderson  (1998) 
to  determine  the  phase  of  social  construction  of  knowledge  for  each  group. 

•  Code  online  dialogue  for  collaboration  utterances  (Curtis  and  Lawson,  1999) 

•  Code  participation  for  surface  or  in-depth  commenting. 

•  Map  changes  in  interactions  over  time. 

Hypotheses: 

1 .  Group  2  (synchronous  generation,  asynchronous  evaluation)  will  be  most  productive  in 
number  of  terms  generated  and  evaluated. 

2.  Group  2  will  also  exhibit  higher  levels  of  social  construction  of  knowledge  than  the  other 
groups. 

Theory: 

Real-time  chat  will  allow  group  members  to  learn  about  each  other  and  generate  many 
ideas,  while  latter  delayed  discussion  will  promote  negotiation  of  those  ideas. 

Experiment  VII.  Peer  Tutoring  and  Online  Mentoring:  Impact  of  Peer  Tutoring  on  E-Learning 
Achievement 

This  study  looks  at  the  impact  of  peer  tutoring  on  student  e-leaming  achievement.  What 
is  effective  timing,  quantity  or  extent,  type,  etc.  of  online  mentoring?  Students  in  the  e-leaming 
course  will  be  randomly  assigned  to  one  of  six  conditions:  (1)  mentoring  by  former  students  who 
successfully  completed  the  course  online;  (2)  mentoring  by  former  students  who  took  the  course 
in  a  live  setting;  (3)  mentoring  by  guest  experts;  (4)  mentoring  by  peers  in  the  current  course;  (5) 
mentoring  from  a  combination  of  peers,  guests,  and  experts;  and  (6)  a  no  mentoring  control 
condition.  All  mentors  will  receive  some  brief  online  training  about  forms  of  online  mentoring 
(see  Table  1 1).  They  will  then  be  available  to  the  students  via  electronic  mail  for  questions, 
advice,  and  general  support.  Student  performance  in  each  environment  will  be  compared. 
Student  preferences  for  these  different  mentoring  environments  will  be  addressed  with  both  a 
questionnaire  and  interviews  of  randomly  selected  students.  Transcripts  will  be  coded  for  the 
types  of  online  mentoring  and  scaffolding  in  each  of  these  environments.  In  addition,  this 
analysis  will  note  the  timing  and  degree  of  online  support  that  was  provided.  A  comparison  of 
the  effectiveness  of  the  five  mentoring  conditions  will  look  at  the  level  and  timing  of  support  as 
well  as  overall  mentor  responsiveness.  Two  of  the  mentors  in  each  condition  will  also  be 
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interviewed  via  the  telephone  or  an  online  chat.  At  the  end  of  this  study,  an  online  mentoring 
guide  will  be  developed  to  help  in  the  selection  and  training  of  mentors.  A  key  hypothesis  here 
is  that  the  combined  mentoring  approach  and  mentoring  by  students  who  took  the  course  in  an 
online  format  will  result  in  higher  levels  of  student  performance.  Second,  students  in  any  of  the 
five  mentoring  conditions  will  outperform  those  in  the  control  condition. 


Experiment:  What  type  of  scheduling,  approach,  quantity,  etc.  of  online  mentoring  is  most 
effective? 


Independent  variable:  Mentoring  condition  (6) 

Dependent  variable:  Student  performance 

1 .  Mentoring  by  former  students  who  have 
successfully  completed  the  course  online 

2.  Mentoring  by  former  student  who  have 
taken  the  course  in  a  live  setting 

3.  Mentoring  by  guest  experts 

4.  Mentoring  by  peers  in  the  current  course 

5.  Mentoring  by  a  combination  of  peers  and 
experts 

6.  Control,  no  mentoring 

•  Compare  student  performance 
between  groups 

*Note:  All  mentors  receive  some  training  on  online  mentoring,  and  use  e-mail  to  communicate 
with  students. 


Follow  up: 

•  Questionnaires  and  interview  of  randomly  selected  students  will  be  used  to  assess  student 
preference  for  different  conditions  (1-6). 

•  Transcripts  will  be  coded  to  include:  Type  of  mentoring  and  scaffolding,  and  timing  and 
degree  of  online  support  provided. 

•  Two  of  the  mentors  from  each  condition  will  be  interviewed. 

•  An  online  mentoring  guide  will  be  developed  to  help  the  selection  of  future  mentors. 
Hypotheses: 

1 .  The  combined  mentoring  approach  and  mentoring  by  students  who  took  the  course  online 
will  yield  the  highest  student  performers. 

2.  Students  in  all  of  the  five  mentoring  conditions  will  outperform  the  control  group. 

Experiment  VIII.  Student  Retention:  E-Learning  and  Attrition 

Though  relying  mainly  on  anecdotal  data,  it  seems  that  a  major  problem  of  e-leaming 
environments  is  the  retention  of  students.  What  factors  help  new  recruits  complete  a  degree 
program?  What  e-leaming  formats  and  approaches  stimulate  greater  motivation?  What 
pedagogical  factors  facilitate  retention?  For  instance,  does  a  more  flexible  format  increase 
retention?  In  this  experiment,  there  are  four  learning  formats  available: 

1 .  instructor  minimal  presence,  extensive  student  choice — in  this  condition,  students  decide 
on  all  assignments  from  a  menu  of  possibilities  provided  by  the  instructor; 

2.  instructor  dictated,  minimal  choice — here,  the  instructor  is  clear  about  all  assignments 
and  provides  explicit  task  structuring; 

3.  instructor  guidance,  some  choice — here  the  instructor  acts  as  a  guide  or  facilitator  of 
student  online  learning  while  students  have  some  input  in  class  activities;  and 
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4.  instructor  varied  roles  in  an  online  community — the  emphasis  in  the  fourth  condition  is 
on  building  collaborative  teams  and  constructing  knowledge  in  groups;  hence,  the  role  of  the 
instructor  is  constantly  changing. 

Student  attitudes  and  attrition  rates  will  be  compared  across  the  four  conditions. 
Computer  logging  data  will  note  student  participation  over  time.  Access  to  technology,  previous 
computer  experience,  and  self-efficacy  will  be  correlated  with  course  attrition.  Transcripts, 
interviews,  and  retrospective  analyses  will  determine  what  communication  patterns  or 
instructional  techniques  are  beneficial  to  students  whose  interaction  wanes  during  different  time 
periods  of  the  course.  Interviews  and  course  content  analyses  with  the  instructors  will  determine 
the  more  and  less  successful  strategies  here.  Upon  completion  of  the  course,  student  intentions 
to  enroll  in  other  e-leaming  situations  will  be  compared.  It  is  hypothesized  that  the  first  two 
conditions  will  result  in  the  highest  dropout  rates. 


Experiment:  What  are  the  pedagogical  factors  that  foster  retention  of  online  students? 


Independent  variable:  Learning  format  (4) 

Dependent  Variable:  Course  attrition 

1 .  Teacher  minimal  presence,  extensive 
student  choice  of  assignments  from  a  menu 

2.  Teacher  dictated,  minimal  student  choice 

3.  Teacher  guidance,  some  student  choice 

4.  Teacher  varied  roles  in  an  online 
community,  teacher  role  is  dynamic 

•  Attrition  rates 

•  Attrition  categories 

•  Student  attitudes 

Follow  up: 

•  Access  to  technology,  computer  experience  and  self-efficacy  will  be  correlated  with  attrition. 

•  Student  intentions  to  enroll  in  future  e-leaming  situation  will  be  investigated. 

Hypothesis: 

1 .  Condition  1  and  2  will  result  in  the  highest  drop-out  rate. 

Experiment  IX.  Conceptual  Referencing:  Collaboration  and  Explicit  Referencing 

How  do  different  collaborative  tools  foster  student  learning  of  key  concepts?  Can  tools 
help  students  explicitly  reference  course  ideas,  concepts,  and  issues  as  well  as  the  ideas  of  their 
peers?  Will  student  explicit  conceptual  and  peer  referencing  in  online  environments  foster 
greater  learning  and  interaction  than  in  traditional  classroom  settings?  In  this  research  project, 
students  will  study  a  set  of  tactical  principles  and  then  practice  them  in  a  simulated  gaming 
environment.  After  the  simulation,  one  group  will  discuss  the  tactics  in  small  groups  of  4-6 
participants  using  a  computer  conferencing  tool.  Here  students  will  be  forced  to  label  their  posts 
as  hypotheses,  evidence,  data,  critical  issues,  personal  opinions,  etc.  They  will  also  be  required 
to  refer  to  at  least  one  previous  post  in  each  of  their  messages.  The  other  group  will  provide  the 
instructor  with  individual  reflection  summaries  of  the  activity  via  email.  The  depth  of  student 
reflections  in  each  environment  will  be  compared.  The  following  week,  another  set  of  similar 
maneuvers  will  be  discussed  in  a  synchronous  chat.  The  ideas  of  students  in  each  group  will  be 
compared.  Student  attitudes  toward  each  environment  will  be  compared.  It  is  hypothesized  that 
students  in  the  small  group  condition  will  outperform  the  individual  reflection  condition  during 
the  synchronous  chats. 
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Experiment:  Two  groups  of  students  will  study  five  key  military  maneuvers  and  practice  them  in 
a  simulated  environment.  _ _ 


Independent  variable:  Group  membership  (2) 

Dependent  variable:  Performance  during  chat 

1 .  Small  group  (4-6)  using  a  computer 
conferencing  tool  such  as,  ACT  to 
discuss  tactics 

2.  Individual  providing  reflective 
summaries  to  an  instructor  via  email 

•  Depth  of  reflections 

•  Strength  of  ideas 

Follow  up: 

•  One  week  later  a  set  of  similar  maneuvers  will  be  discussed  in  a  synchronous  chat. 
Hypothesis: 

1 .  Students  in  the  small  group  condition  will  outperform  the  individual  reflection  condition 
during  the  synchronous  chats. 

Experiment  X.  Online  Collaboration:  Group  Performance  and  Collaborative  Learning 
Technologies 

As  collaborative  tools  are  developed,  it  is  vital  to  understand  how  groups  function  and 
perform  online.  How  do  members  share  different  views  of  data  and  instrument  controls?  How 
do  text  pointing  and  document  sharing  devices  impact  on  collaboration?  What  cutting  edge 
collaborative  learning  technologies  might  foster  student-centered  learning?  How  aware  are 
students  and  instructors  of  the  various  tools  for  collaboration?  For  this  study,  the  Army  will 
acquire  a  set  of  ten  or  more  different  collaborative  tools  for  e-leaming.  Once  acquired,  50 
individuals  will  be  taken  to  a  computer  lab  wherein  they  will  receive  demonstration  on  different 
types  of  collaborative  tools  and  activities  (e.g.,  conferencing  tools,  videoconferencing,  email, 
collaborative  writing  and  notetaking  tools,  peer  writing  document  annotation  tools,  chat  tools, 
group  decision  support  systems,  voting  and  polling  tools,  electronic  brainstorming,  etc.).  They 
will  then  meet  together  as  a  focus  group  to  discuss  the  benefits  and  limitations  of  some  of  these 
tools.  They  will  then  rate  these  tools  for  the  degree  of  collaboration,  ease  of  use,  degree  of 
interaction,  and  potential  for  online  course  adaptability.  In  a  follow-up  experiment,  a  second 
focus  group  of  5-6  collaborative  learning  experts  will  evaluate  these  features  and  make 
suggestions  for  improvement.  Here,  a  nominal  group  process  will  be  used  to  gather  initial 
opinions  and  rankings.  In  the  third  leg  of  this  study,  student  and  instructor  preferences  will  be 
compared  to  their  learning  style  inventories.  Student  and  expert  collaborative  tool  preferences 
also  will  be  compared. 


Initial  study:  Have  students  compare  collaborative  tools  for  e-leaming. 


Focus  group:  Discuss  benefits  and 
limitations  of  20-30  different  tools 

Rating  the  tools 

Examples  of  tools: 

•  Videoconferencing 

•  Email 

•  Collaborative  writing  and  note  taking 

•  Chat 

•  Group  decision  support  systems 

•  Voting 

Rating  dimensions: 

•  Degree  of  collaboration 

•  Ease  of  use 

•  Degree  of  interaction 

•  Potential  for  online  course  adaptability 
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Follow  up: 

•  Focus  group  of  5-6  collaborative  learning  experts. 

•  Give  opinions,  evaluate  features  and  make  suggestions. 

•  Compare  student  and  expert  tool  preferences  to  their  learning  style  inventories. 

•  Compare  student  and  expert  collaborative  tool  preferences. 

Suggested  Secondary  Experiments 

Of  course,  there  are  many  other  possible  experiments.  Seventeen  additional  experiments 
are  briefly  described  in  Appendix  C.  The  purpose  is  to  illustrate  the  wealth  of  opportunities  for 
the  Army  to  capitalize  on  concepts,  experiments,  and  findings  from  the  educational  literature. 
Many  more  could  have  been  designed. 


New  Tools  and  Pedagogies 

As  we  look  to  the  future,  it  is  clear  that  e-leaming  pedagogy  and  research  is  just 
beginning  to  be  defined  and  understood.  There  are  emerging  opportunities  for  encouraging 
cross-classroom  collaboration,  guiding  online  mentoring  and  apprenticeship,  facilitating 
telepresence  events  and  activities,  offering  professional  development,  using  archived  information 
databases,  and  asking  questions  of  online  human  information  sources  (Riel  &  Harasim,  1994). 
Teamwork  on  projects  with  real  audiences  or  clients  across  the  globe  is  now  a  common  way  to 
engage  students  in  the  learning  process  (Bonk  &  Sugar,  1998;  Kearsley  &  Shneiderman,  1998; 
Riel,  1990). 

However,  tools  for  fostering  such  team  projects  and  interactions  are  crude  or  nonexistent 
in  most  higher  education  and  training  courseware  today  (Bonk  &  Dennen,  1999).  Clearly,  there 
is  a  need  to  develop  collaborative  tools  such  as  domain  specific  whiteboards  that  help  create, 
edit,  and  display  drawings,  graphs,  and  models.  In  addition,  annotation  and  sharing  tools  with 
advanced  visualizations  and  modeling  tools  should  populate  the  e-leaming  workspace  of  the  near 
future  (Roschelle  &  Pea,  1999).  Some  of  these  tools  and  computer  systems  will  prestructure  the 
learning  contributions  and  interactions,  while  others  will  allow  more  open-ended  inquiry.  Tools 
already  exist  to  support  multiple  viewpoints,  reflection,  frequent  feedback,  knowledge 
construction  and  notetaking,  shared  representations,  and  progressive  expert  questioning 
(Roschelle  &  Pea,  1 999).  A  myriad  of  technologies  are  available  to  match  different  tasks  and 
individual  needs.  For  example,  student  collaboration  can  be  fostered  via  videoconferencing, 
computer  conferencing,  e-mail,  document  sharing,  and  document  authoring  (Holland,  1996). 

In  light  of  these  technological  developments,  it  is  vital  to  reflect  on  how  to  develop  online 
interactions  with  these  tools  and  how  to  foster  social  negotiation  of  meaning  (Bonk,  1998; 

Hughes  &  Hewson,  1998;  Parker,  1999).  Instructors  not  only  have  to  think  carefully  about  the 
activity  structures  and  instructional  events  that  they  want  to  include,  but  they  now  must  evaluate 
available  technologies  for  each  new  structure  or  pedagogical  strategy  that  they  dream  up.  As 
technologies  for  interaction  advance  at  rates  unimaginable  a  mere  decade  or  even  a  few  months 
ago,  there  is  a  need  to  test  pedagogical  inventions  that  can  foster  online  collaboration  and 
problem  solving  such  as  case  studies,  jigsaw,  brainstorming,  role  plays,  simulations, 
symposiums,  delphi  techniques,  transcript  based  assignments,  forums,  nominal  group  techniques, 
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projects,  panels,  etc.  (Bonk  &  Dennen,  1999;  McLoughlin  &  Oliver,  1999;  Paulsen,  1995b). 

How  might  e-leaming  courses  be  enhanced,  extended,  or  transformed  with  global  interpersonal 
exchanges,  electronic  guest  experts,  collaborative  databases,  information  exchanges,  electronic 
publication  of  student  work,  extensive  peer  feedback,  or  online  simulations  (Harris,  1998)? 

When  will  such  pedagogical  experiments  and  successes  be  shared  and  discussed?  And  who  will 
foster  this  sharing  of  online  course  information? 

The  issues  and  questions  raised  above  will  only  accelerate  in  the  near  future.  Already, 
McKenzie,  Kirby,  Newbill,  and  Davidson  (1998,  p.  123)  contend,  “As  the  world  grows  smaller 
the  need  for  more  educational  opportunities  for  more  people  in  more  places  will  become 
epidemic.  Effective  distance  educational  programs  led  by  effective  instructional  design 
specialists  will  have  to  one  of  the  cures.”  As  that  happens,  education  and  training  in  the  US 
military  will  be  increasingly  defined  as  collaborative  and  leamer-centered  in  nature.  Research 
and  experimentation  on  emerging  tools  and  pedagogies  will  be  more  vital  than  ever  before.  As  a 
result,  the  research  performed  by  the  Army  on  collaborative  and  e-leaming  tools  from  a  student- 
centered  learning  perspective  today  will  play  a  significant  role  in  improving  learning  gains 
tomorrow. 

As  the  Army  transforms  from  a  classroom-centric  to  a  soldier-centric  model  of 
instruction,  planners,  training  developers,  and  researchers  must  maintain  cognizance  of  the  14 
leamer-centered  psychological  principles  outlined  in  Table  1.  For  the  full  benefits  of  the  new 
e-leaming  and  collaborative  tools  emerging  from  the  educational  marketplace,  conformance  to 
these  principles  and  planned  adaptation  to  a  military  training  environment  are  key  factors  for 
success.  Hopefully,  the  sources  provided  in  this  report  and  the  experiments  outlined  will  provide 
a  complementary  impetus  to  the  transformation. 
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Appendix  A 
Table  1 

Learner-Centered.  Psychological  Principles  Revised 


Cognitive  and  Metacognitive  Factors 

1 .  Nature  of  the  learning  process.  The  learning  of  complex  subject  matter  is  most  effective  when 
it  is  an  intentional  process  of  constructing  meaning  from  information  and  experience. 

2.  Goals  of  the  learning  process.  The  successful  learner,  over  time  and  with  support  and 
instructional  guidance,  can  create  meaningful,  coherent  representations  of  knowledge. 

3.  Construction  of  knowledge.  The  successful  learner  can  link  new  information  with  existing 
knowledge  in  meaningful  ways. 

4.  Strategic  thinking.  The  successful  learner  can  create  and  use  a  repertoire  of  thinking  and 
reasoning  strategies  to  achieve  complex  learning  goals. 

5.  Thinking  about  thinking.  Higher  order  strategies  for  selecting  and  monitoring  mental 
operations  facilitate  creative  and  critical  thinking. 

6.  Context  of  learning.  Learning  is  influenced  by  environmental  factors,  including  culture, 

technology,  and  instructional  practices. _ 


Motivational  and  Affective  Factors 

7.  Motivational  and  emotional  influences  on  learning.  What  and  how  much  is  learned  is 
influenced  by  the  learner’s  motivation.  Motivation  to  learn,  in  turn,  is  influenced  by  the 
individual's  emotional  states. 

8.  Intrinsic  motivation  to  learn.  The  learner's  creativity,  higher  order  thinking,  and  natural 
curiosity  all  contribute  to  motivation  to  learn.  Intrinsic  motivation  is  stimulated  by  tasks  of 
optimal  novelty  and  difficulty,  relevant  to  personal  interests,  and  providing  for  personal  choice 
and  control. 

9.  Effects  of  motivation  on  effort.  Acquisition  of  complex  knowledge  and  skills  requires  extended 
learner  effort  and  guided  practice.  Without  the  learner's  motivation  to  learn,  the  willingness  to 
exert  this  effort  is  unlikely  without  coercion. 


Developmental  and  Social  Factors 

10.  Developmental  influences  on  learning.  As  individuals  develop,  there  are  different  opportunities 
and  constrains  for  learning.  Learning  is  most  effective  when  differential  development  within  and 
across  physical,  intellectual,  emotional,  and  social  domains  is  taken  into  account. 

11.  Social  influences  on  learning.  Learning  is  influenced  by  social  interactions,  interpersonal 
relations,  and  communication  with  others. 


Individual  Differences 

12.  Individual  differences  in  learning.  Learners  have  different  strategies,  approaches,  and 
capabilities  for  learning  that  are  a  function  of  prior  experience  and  heredity. 

13.  Learning  and  diversity.  Learning  is  most  effective  when  differences  in  learners'  linguistic, 
cultural,  and  social  backgrounds  are  taken  into  account. 

14.  Standards  and  assessment.  Setting  appropriately  high  and  challenging  standards  and  assessing 
the  learner  as  well  as  learning  progress — including  diagnostic,  process,  and  outcome 
assessment — are  integral  parts  of  the  learning  process. 


A-l 


Table  7 

Indicators  of  critical  thinking  for  online  discourse 
(from  Newman,  Johnson,  Cochrane,  &  Webb,  1 996 ) 


R+/-  Relevancy 

R+  Relevant  Statements 

R-  Irrelevant  statements,  diversions 

1+/-  Importance 

1+  Important  points/issues 

I-  Unimportant,  trivial  points/issues 

N+/-  Novelty,  new 
info,  ideas, 
solutions 

N+  New  problem-related  information 

N-  Repeating  what  has  been  said 

N+  New  ideas  for  discussion 

N-  False  or  trivial  leads 

NS+  Accepting  first  offered  solution 

NS-  Accepting  first  offered  solution 

NQ+  Welcoming  new  ideas 

NQ-  Squashing,  putting  down  new  ideas 

NL+  Learner  (student)  brings  new  things 
in 

NL-  Dragged  in  by  tutor 

0+/-  Bringing 
outside 

knowledge/experie 
nee  to  bear  on 
problem 

0E+  Drawing  on  personal  experience 

0C+  Refers  to  course  material 

0M+  Use  relevant  outside  material 

0K+  Evidence  of  using  previous 
knowledge 

0P+  Course  related  problems  brought  in; 
e.g.,  students  identify  problems  from 
lectures  and  texts 

0Q+  Welcoming  outside  knowledge 

OQ-  Squashing  attempts  to  bring  in 
outside  knowledge 

0-  Sticking  to  prejudice  or  assumptions 

A+/-  Ambiguities; 
clarified  or 
confused 

AC+  Clear,  unambiguous  statements 

AC-  Confused  statements 

A+  Discuss  ambiguities  to  clear  them  up 

A-  Continue  to  ignore  ambiguities 

L+/-  Linking 
ideas, 

interpretation 

L+  Generating  new  data  from 
information  collected 

L-  Repeating  information  without  making 
inferences  or  offering  an  interpretation 

L+  Linking  facts,  ideas,  and  notions 

L-  Stating  that  one  shares  the  ideas  or 
opinions  stated,  without  taking  these 
further  or  adding  any  personal  comments 

]+/-  Justification 

JP+  Providing  proof  or  examples 

JP-  Irrelevant  or  obscuring  questions  or 
examples 

JS+  Justifying  solutions  or  judgments 

JS-  Offering  judgments  or  solutions 
without  explanations  or  justification 

JS+  Setting  out  advantages  and 
disadvantages  of  situations  or  solution 

JS-  Offering  several  solutions  without 
suggesting  which  is  the  most  appropriate 

C+/-  Critical 
assessment 

C+  Critical  assessment/evaluation  of 
own  or  others  contributions 

C-  Uncritical  acceptance  or  unreasoned 
rejection 

CT+  Tutor  prompts  for  critical 
evaluation 

CT-  Tutor  uncritically  accepts 

P+/-Practical 
utility  (grounding) 

P+  Relate  possible  solutions  to  familiar 
situations 

P-  Discuss  on  a  vacuum  (treat  as  if  on 

Mars) 

P+  Discuss  practical  utility  of  new  ideas 

P-  Suggest  impractical  solutions 

W+/-  Width  of 
understanding 
(complete  picture) 

W+  Widen  discussion;  e.g.,  problem 
within  a  larger  perspective,  intervention 
strategies  within  a  wider  framework 

W-  Narrow  discussion;  e.g.,  address  bits 
or  fragments  or  situation,  suggest  glib, 
partial,  interventions 

Table  8 

Constructivist  Interaction  Analysis  Coding  Scheme 
(Kanuka  &  Anderson,  1998) 


_ Phase  I:  Sharing/Comparing  of  Information _ 

a.  A  statement  of  observation  or  opinion 

b.  A  statement  of  agreement  from  one  or  more  other  participants 

c.  Corroborating  examples  provided  by  one  or  more  participants 

d.  Asking  and  answering  questions  to  clarify  details  of  statements 

e.  Definition,  description  or  identification  of  a  problem  _ 

_ Phase  II:  Discovery  of  dissonance  and  inconsistency _ 

a.  Identifying  and  stating  areas  of  disagreement 

b.  Asking  and  answering  questions  to  clarify  disagreement 

c.  Restating  and  possibly  advancing  arguments  in  its  support _ 

_ Phase  III.  Negotiation  of  Meaning/Co-construction  of  knowledge 

a.  Negotiation  or  clarification  of  meaning  of  terms 

b.  Negotiation  of  weight  assigned  to  types  of  argument 

c.  Negotiation  of  agreement  among  conflicting  concepts 

d.  Negotiation  of  compromise  or  co-constructions 

e.  Proposal  of  integrating/accommodating  metaphors  or  analogies _ 

_ Phase  IV.  Testing  and  modification  of  proposed  synthesis 

a.  Testing  that  proposal  synthesis 

b.  Testing  against  existing  cognitive  schema 

c.  Testing  from  personal  experience 

d.  Testing  against  informal  data  collected 

e.  Testing  against  contradictory  testimony  in  the  literature _ 

_  Phase  V.  Agreement/application  of  newly  constructed  meaning 

a.  Summarization  of  agreement 

b.  Illustrations  of  the  new  knowledge  as  applied  to  work 

c.  Metacognitive  statements  by  the  participants  illustrating  change _ 
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Table  9 

Coding  scheme  to  describe  utterances  in  online  collaboration 
(Curtis  &  Lawson,  1999) 


Behavior 

Categories 

Codes 

Description 

Example 

Planning 

GS 

Group  skills;  a  generic  code 
applied  to  expressions  that 
encourage  group  activity  and 
cohesiveness 

I  know  that  [names]  have  given  you 
good  advice,  but  I  think  it’s  worth 
knowing  that  you  need  patience. 

OW 

Organizing  work:  Planning  group 
work;  setting  shared  tasks  and 
deadlines. 

I  just  want  to  set  a  time-line  for 
myself.  Is  everyone  OK  with  that? 

IA 

Initiating  activities:  Setting  up 
activities  such  as  chat  sessions  to 
discuss  the  progress  and 
organization  of  group  work. 

I  would  like  to  chat  on  the  blackboard. 
What  about  this  Friday  at  7:30  pm  SA 
time? 

Contributing 

HeG 

Help  giving:  Responding  to 
questions  and  requests  from  others. 

To  access  the  chat  room,  click  on 
virtual  chat  in  the  blackboard;  chat 
screen  will  come  on;  click  on  enter. . . 

FBG 

Feedback  giving:  Providing 
feedback  on  proposals  from  others. 

I  like  your  idea  of  a  generic  booklet 
and  everyone  contributing  aspects  of 
interesting  internet  services... 

RI 

Exchanging  resources  and 
information  to  assist  other  group 
members 

With  the  implementation  of  an  internet 
service. .  .there  has  been  a  major  shift 
in  the  communication  function  in 
business. 

SK 

Sharing  knowledge:  Sharing 
existing  knowledge  and 
information  with  others. 

I  think  we  also  need  to  give  thought  to 
the  following:  1.  The  issues  of 
quality/efficiency  in  teaching  and 
learning... 

CH 

Challenging  others:  Challenging 
the  contributions  of  other  members 
and  seeking  to  engage  in  debate. 

No  examples — behavior  not  identified 
in  the  text. 

EX 

Explaining  or  elaborating: 
Supporting  one’s  own  position 
(possibly  following  a  challenge). 

No  examples — behavior  not  identified 
in  the  text. 

Seeking 

Input 

HeS 

Help  seeking:  Seeking  assistance 
from  others. 

Does  anyone  know  how  to 
edit/add/append  data  on  the  student 
pages? 

FBS 

Feedback  seeking:  Seeking 
feedback  to  a  position  advanced. 

What  do  you  think  about  answering  the 
questions  that... have  been  put 
forward? 

Ef 

Advocating  effort:  Urging  others 
to  contribute  to  the  group  effort. 

Haven’t  heard  from  you  for  awhile. 

Are  you  still  with  us? 

Reflection/ 

Monitoring 

ME 

Monitoring  group  effort: 

Comments  about  the  group’s 
processes  and  achievements. 

1  believe  the  overall  contribution  and 
collaboration  of  working  as  a  group 
requires  an  increase  within  itself  as 
part  of  our  learning. 

RM 

Reflecting  on  medium:  Comments 
about  the  effectiveness  of  the 
medium  in  supporting  group 
activities. 

The  email  for  the  discussion  group 
seems  to  work  OK  for  me.  You  know 
it  has  gone  through  because  you 
actually  receive  your  email  back 
almost  straight  away  if  it  has  worked. 

Social 

Interaction 

SI 

Social  interaction:  Conversation 
about  social  matters  that  are 
unrelated  to  the  group  task.  This 
activity  helped  to  ‘break  the  ice.’ 

Regarding  chat — my  weekend  is  pretty 
hectic — I  have  my  family  flying  in 
from  Greece. .  .so  the  Greek  festivities 
will  be  in  full  swing. 
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Table  1 1 

Twelve  forms  of  electronic  learning  mentoring  and  assistance 
(Bonk  &  Kim,  1998;  Tharp,  1993) 


1. 

Social  (and  cognitive)  Acknowledgement:  "Hello...,"  "1  agree  with  everything  said  so  far...,"  "Wow, 
what  a  case."  "This  case  certainly  has  provoked  a  lot  of  discussion...,'  Glad  you  could  join  us... 

2. 

Questioning:  "What  is  the  name  of  this  concept...?,"  "Another  reason  for  this  might  be...?,"  An  example 
of  this  is...,"  "In  contrast  to  this  might  be...,""What  else  might  be  important  here...?,"  "Who  can  tell . 
me  7  ”  "How  might  the  teacher..?."  "What  is  the  real  problem  here...?,"  "How  is  this  related  to...?.  Can 

you  justify  this???" 

3. 

Direct  Instruction:  "1  think  in  class  we  mentioned  that...,"  "Doesn't  Chapter  X  talk  about..., 

"Remember  back  to  the  first  week  of  the  semester  when  we  went  over  "X"  which  indicated  that... 

4. 

Modeling/Examples:  "1  think  I  solved  this  sort  of  problem  once  when  I...,"  "Remember  that  video  we 
saw  on  "X"  wherein  "Y"  decided  to...,"  "Doesn't  "X"  give  insight  into  this  problem  in  case  "#..."  when 
he/she  said..." 

5. 

Feedback/Praise:  "Wow,  I'm  impressed...,"  "That  shows  real  insight  into....  Are  you  sure  you  have 
considered...,"  "Thanks  for  responding  to  "X"...,"  "1  have  yet  to  see  you  or  anyone  mention..." 

6. 

Cognitive  Task  Structuring:  "You  know,  the  task  asks  you  to  do...,"  "Ok,  now  summarize  the  peer 
responses  you  have  received...,"  "How  might  Slavin  or  Woolfolk  have  solved  this  case 

7. 

Cognitive  Elaborations/Explanations:  "Provide  more  information  here  that  explains  your  rationale, 

"Please  clarify  what  you  mean  by..."  “I'm  just  not  sure  what  you  mean  by...,"  "Please  evaluate  this  solution 
a  little  more  carefully." 

8. 

Push  to  Explore:  "You  might  want  to  write  to  Dr.  "XYZ"  for...,"  "You  might  want  to  do  an  ERIC  search 
on  this  topic...,"  "Perhaps  there  is  a  URL  on  the  Web  that  addresses  this  topic... 

9. 

Fostering  Reflection/Self  Awareness:  "Restate  again  what  the  teacher  did  here...,"  "How  have  you  seen 
this  before...,"  "When  you  took  over  this  class,  what  was  the  first  thing  you  did...,"  "Describe  how  your 
reaching  philosophy  will  vary  from  this...,"  "How  might  an  expert  teacher  handle  this  situation... 

10. 

Encouraging  Articulation/Dialogue  Prompting:  "What  was  the  problem  solving  process  the  teacher 
faced  here...,"  "Does  anyone  have  a  counterpoint  or  alternative  to  this  situation...,"  "Can  someone  give  me 

3  good  reasons  why...,"  "It  still  seems  like  something  is  missing  here,  I  just  can't  put  my  finger  on  it..." 

ii. 

General  Advice/Scaffolding/Suggestions:  "If  I  were  in  her  shoes,  I  would...,"  "Perhaps  I  would  think 
twice  about  putting  these  kids...,"  "I  know  that  1  would  first...,"  "How  totally  ridiculous  this  all  is;  certainly 
the  teacher  should  be  able  to  provide  some..." 

12. 

Private  E-mail  or  Discussion  Management:  "Don't  just  criticize....please  be  sincere  when  you  respond 
to  your  peers,"  "If  you  had  put  your  case  in  on  time,  you  would  have  gotten  more  feedback.  If  you  do  this 
again,  we  will  have  to  take  away  your  privileges." _ _ _ - 
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Table  12 

Social  Constructivism  and  Learning  Communities  Online  (SCALCO)  Scale  (Bonk, 
Malikowski,  Angeli,  &  East,  1998;  Bonk,  Oyer,  &  Medury,  1995; 

Kanuka  &  Anderson,  1998) 


Part  A.  Social  Constructivism  and  Learning  Communities  Online  Questionnaire 
Questions  (1  =  strongly  agree;  5  =  strongly  disagree) _ 

1.  The  topics  discussed  online  had  real  world  relevance. 

2.  The  online  dialogue  dealt  with  original  topics. 

3.  As  the  forum  progressed,  I  developed  a  position  on  various  topics  that  I  did  not  have  before 
the  online  forum. 

4.  The  online  forum  dialogue  offered  multiple  perspectives. 

5.  The  online  dialogue  encouraged  me  to  reflect  on  the  issues. 

6.  1  integrated  new  knowledge  acquired  from  the  online  discussion  into  my  existing  knowledge, 
which  resulted  in  a  deeper  understanding  of  the  issues. 

7.  I  made  new  connections  to  the  course  material  as  a  result  of  the  online  environment. 

8.  I  have  more  ideas  that  I  can  use  about  this  topic  than  without  the  online  forum. 

9.  The  online  forum  nurtured  my  critical  thinking  and  evaluation  skills. 

10.  I  had  a  voice  within  the  discussion  forum. 

1 1.  I  had  some  personal  control  over  course  activities  and  discussion. 

12.  Online  discussions  were  not  relevant  to  my  learning  needs. 

13.  The  online  technology  allowed  me  to  design  and  create  new  ideas. 

14.  The  online  environment  encouraged  me  to  question  ideas  and  perspectives. 

15.  I  liked  collaborating  with  others  online. 

16.  Instructors  provided  useful  advice  and  guidance  online. 

17.1  could  count  on  others  to  reply  to  my  needs. 

18.  The  online  environment  fosters  an  atmosphere  where  more  than  one  answer  may  be  correct. 

19.  I  collaborated  with  other  participants  in  the  forum  that  resulted  in  new  perspectives  and  a 
better  understanding. 

20.  I  felt  that  I  was  a  member  of  the  group. 

21.  The  other  group  participants  acknowledged  my  contribution  to  the  discussion. 

22.  I  felt  committed  with  other  online  participants  to  work  together  in  order  to  acquire  a  deeper 
understanding  of  the  issues. 

23.  I  felt  the  discussion  took  the  issues  to  a  deeper  level. 

24.  The  online  forum  provided  opportunities  for  in-depth  discussion. 

25.  I  clarified  my  ideas  by  sharing  them  with  others  online. 

26.  I  clarified  my  ideas  by  reading  other  participants’  comments. 

27.  I  gained  an  appreciation  for  other  opinions  and  perspectives. 

28.  I  received  useful  mentoring  and  feedback  from  others. 

29.  The  online  environment  fostered  peer  interaction  and  dialogue  about  real-life  problems. 

30.  The  online  discussions  lowered  the  isolation  and  loneliness  of  similar  learning  situations. 

31.  The  online  forum  fostered  a  sense  of  a  collaborative  learning  community. 

32.  There  was  a  sense  of  membership  in  a  learning  here. 

33.  Other  participants  and  I  make  decisions  about  how  we  will  proceed  or  leam  online. 

34.  Instructors  or  moderators  provide  just  enough  resources  to  help  me  succeed  online. 

35.  This  environment  had  opportunities  to  prepare  answers  with  peers  or  learning  teams. 

36.  Peer  evaluation  and  feedback  was  integrated  into  this  learning  environment. 

37.  The  online  environment  allowed  for  the  exploration  of  topics  of  personal  interest. 

38.  I  could  share  and  discuss  my  ideas  and  answers  with  others  in  this  environment. 

39.  It  was  interesting  to  see  how  differences  of  opinion  were  discussed  and  negotiated  in  this 
environment. 

40.  Summaries  or  compromise  positions  were  facilitated  in  this  environment. _ 
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Appendix  B 

List  of  URLs  (Universal  Resource  Locators)  cited  in  text 


Page  5 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  FY200  program 
(http://www.ari.army.mil) 

Page  8 

Learner-Centered  Psychological  Principles  Revised  (http://www.apa.or/ed/lcp.html) 
Page  10 

Carr,  2000a,  February  11th  (http://chronicle.com/free/v46/i23/23a00101.htm) 

Page  13 

Peffers  &  Bloom,  1999  (http://clam.rutgers.edu/~ejoumal/spring99/survey.htm) 

Paulsen,  1 995a  (http://www.emoderators.com/moderators/morten.html) 

Page  17 

Bannon  &  Hughes, 1993  (http://www.ul.ie/~idc/library/papersreports/LiamBannon/ 

1/BannonHughes.html) 

CSCL  Conference  1995  (http://www-cscl95.indiana.edu/cscl95) 

Page  18 

Lau  &  Hayward,  1997  (http://search.ahfmr.ab.ca/tech_eval/gss.htm) 

Page  23 

Paulsen,  1995b  (http://www.hs.nki.no/~morten/cmcped.htm) 

Page  24 

Herring,  1999  (http://www.ascusc.org/jcmc/vol4/issue4/herring.html) 

Page  3 1 

TAPPED  IN  (http://www.tappedin.sri.com) 

Page  34 

Russell,  1999  (http://cuda.teleeducation.nb.ca/nosignificantdifference/) 

Page  35 

WebTrends  Log  Analyzer  software  (http://www.webtrends.com) 

Carr,  2000b,  February  14th  (http://chronicle.com/free/2000/02/200021401u.htm) 

Page  37 

College  syllabi  posted  at  the  World  Lecture  Hall  (http://www.utexas.edu/world/lecture/) 
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Appendix  C 
Secondary  Experiments 


1.  Testing  and  comparison  of  the  effectiveness  of  various  instructional  approaches  (i.e., 
debates,  role  play,  synchronous  chatting,  online  forums,  online  symposia,  simulations, 
etc.)  (Doherty,  1998).  Which  of  these  pedagogical  activities  are  more  likely  to  stimulate 
student  motivation  and  satisfaction  and  knowledge  acquisition?  (Bracewell,  Breuleux, 

Laferriere,  Benoit,  &  Abdous,  1998). _  ' _ 

Independent  variables:  Demographics,  two  or  more  instructional  approaches,  two  or 

more  learning  categories,  frequency  of  use  (e.g.,  daily,  weekly,  monthly) _ 

Dependent  variables:  Performance  on  post-test,  level  of  satisfaction,  ease  of  use _ 

Application  area:  Captains  Career  Course,  Professional  Development,  Reserve 
Component  functional  training _ _ _ 

2.  Does  participation  in  e-leaming  increase  writing  skills,  problem  solving  skills,  and 

knowledge  retention?  (Riel  &  Harasim,  1984). _ 

Independent  variables:  Instructional  approach:  conventional  vs.  DL,  a  writing  sample, 

demographics  _ _ _ 

Dependent  variables:  Performance  on  multiple  post-tests,  writing  sample,  metacognitive 

skills  demonstrated  in  a  problem  solving  task. _ _ 

Application  area:  Service  Academies;  Professional  Development _ 

3.  What  is  the  balance  between  presentational  and  constructivist  pedagogical  strategies 

for  different  types  of  problem  solving  or  e-leaming?  (Dede,  1996b) _ 

Independent  variables:  2  or  more  instructional  approaches,  2  pedagogical  strategies, 

demographics  _ _ _ 

Dependent  variables:  Metacognitive  strategies  used,  problem  solving  task _ 

Application  area:  Digital  skill  training;  Battle  Staff  _ 

4.  What  types  of  learning  assistance  and  support  do  peers  and  expert  mentors  provide  in 
Web-based  conferences?  (Bonk,  Malikowski,  Angeli,  &  East,  1998;  Bonk,  Malikowski, 

Angeli,  &  Supplee,  1998;  See  Table  11) _ _ _ 

Independent  variables:  2+  types  of  conference  formats,  support  personnel  (peer  and 

expert),  demographics _ _ _ 

Dependent  variables:  Feedback  and  self  report:  Was  assistance  adequate?  Which  support 

person  helped  you  the  most?  Availability  of  support? _ 

Application  area:  Development  of  operation  order;  Web  conferencing _ 

5.  How  might  student  learning  styles  be  utilized  and  perhaps  mixed  in  online 

collaborative  teams  to  maximize  interaction  and  performance? _ 

Independent  variables:  Learning  strategies  (homogeneous  and  heterogeneous  groups), 

demographics _ _ _ 

Dependent  variables:  Post-test  measure  of  performance,  attitudinal  measure,  productivity, 

multiple  post-tests _ _ _ _ _ 

Application  area:  Throughout  Army  Distance  Learning;  Refresher  training _ 
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6.  Comparison  of  computer  adaptive  testing  with  different  levels  of  hints  or  scaffolds 

available  to  the  learners  on  demand  (i.e.,  from  general  to  specific). _ 

Independent  variables:  Demographics,  3  groups:  1)  with  all  hints,  2)  with  solicited  hints 
only,  3)  with  guided  hints  (artificial  intelligence  or  hints  for  different  levels  of  needs), 

specificity  of  hints  (general,  detailed) _ _ _ 

Dependent  variables:  Performance,  attitudes/satisfaction,  learning  styles _ 

Application  area:  Refresher  training;  Individual  Ready  Reserve _ 

7.  Who  decides  when  a  skill  is  needed  as  well  as  who  should  provide  the  training?  Some 
insight  might  come  from  a  study  of  learner  search  strategies  and  navigation  when  helping 
solve  a  team  project?  What  increases  search  effectiveness,  depth,  and  satisfaction?  How 

do  other  team  members  encourage  exploration? _ 

Independent  variables:  Demographics,  computer/Intemet  experience,  previous  use  of 

search  engines,  measure  of  open-mindedness,  computer  anxiety,  self-efficacy _ 

Dependent  variables:  Group  work  survey,  resourcefulness  (#  of  hits),  performance  on 

selected  search  tasks  (e.g.,  time,  efficiency,  comprehension,  problem  solving) _ 

Application  area:  University  After  Next;  Operational  USCRS  (S-3) _ 

8.  Comparison  of  student  attitudes  and  experiences  when  they  know  their  work  has  real 
or  authentic  audiences  and  will  be  archived  for  future  learners  (i.e.,  permanent)  versus 
those  in  environments  wherein  their  audience  is  simply  the  teacher  (impermanent  text). 
Independent  variables:  Demographics,  experience  level  with  subject  matter,  2  groups 
(permanent  vs.  impermanent  text),  goal  orientation — task  completion  or  learning  or 

mastery  _ _ _ 

Dependent  variables:  Attitudes,  Post-test  measure  of  performance  _ 

Application  area:  Professional  Development;  Digital  Libraries _ 

9.  Comparisons  of  the  types  of  scaffolding  instructors  employ  in  procedural  and  more 

complex  training  environments. _ _ _ 

Independent  variables:  Demographics,  2  groups  (complex  vs.  procedural  training 

environments),  prior  experience  with  procedural  and  or  complex  training _ 

Dependent  variables:  Post-test  of  performance,  self-reported  perceptions  of  professor, 
Application  area:  BNCOC;  OBC _ _ _ 

10.  The  development  of  a  procrastination  questionnaire  as  a  screening  mechanism  for 

students  wanting  to  take  an  e-leaming  course.  _ _ _ 

Independent  variables:  Self-proclaimed  procrastinators  vs.  non -procrastinators  (score  on 
10-20  item  survey  of  procrastination),  previous  experience  with  DL,  other  individual 
difference  variables  (self-efficacy,  locus  of  control),  number  of  DL  courses  taken  in  the 

past,  demographics _ _ _ 

Dependent  variables:  Desire  for/likelihood  of  taking  a  DL  course  in  the  future, 
performance,  degree  to  which  you  procrastinated  in  previous/current  course  -DL  or  trad. 
Application  area:  Reserve  Component;  Reclassification  training  _ 

11.  What  online  environmental  supports  foster  conversations  and  shared  explorations  that  [ 
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Application  area:  Professional  Development;  Web-based  training  applications 


15.  Are  there  variations  in  the  content  and  length  of  the  dialogue  for  students  receiving 
different  levels  of  scaffolding  in  e-learning?  (Bonk,  Malikowski,  Angeli,  &  East,  1998) 
Independent  variables:  Groups  receiving  different  levels  of  scaffolding  (none,  some,  a 

lot),  demographics,  amount  of  dialogue  experienced  in  the  past _ 

Dependent  variables:  Measure  of  dialogue:  content,  length,  effectiveness, 
appropriateness  _ 


Application  area:  Web-based  training;  Synchronous  trainin 


16.  How  extensively  do  students  use  online  practice  tests  and  does  this  correlate  with 
success  after  factoring  out  preexisting  differences  in  metacognitive  ability  and/or 
motivation?  _ 
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Independent  variables:  Learning  styles,  goal  orientation,  metacognition,  motivation, 

computer  anxiety,  self-efficacy,  other  individual  differences _ 

Dependent  variables:  Use  of  online  practice  tests,  performance  measure/success 
Application  area:  BNCOC;  Reserve  Component _ 


17.  Classify  the  cognitive  (introduction  to  the  problem,  compare  positions,  draw 
conclusions,  challenge  solutions  argue  positions,  etc.)  and  metacognitive  acts 
(monitoring,  evaluation,  planning,  etc.)  of  problem  solving  in  small  groups.  Which  skills 

are  more  prevalent  and  why? _ 

Independent  variables:  Heterogeneous  groups,  demographics _ 

Dependent  variables:  Group  work  survey,  self-assessment  of  cognitive  and 

metacognitive  tasks  used, _ 

Application  area:  Battle  Staff _ 
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